••  lUKHi*  CUUi»<AtiO«l 

unclassified 


a*  UCU«<TV  CiASiifKATCM 


a»  OCCiASSiriCATlON/ 


4  MMORMinG  O4CAMlZATI0«t 


KEPOKT  D0CUMCNTAT10N  MCC 


14  UiTK<fivt  MAAii.i«>C) 


OlSTRlIUTiQMfAVAtUUlUTT  0(  RtRQRT  - 

Approved  for  public  release; 
Unlimited  distribution. 


S  MONlTOKnuO  ORCANUAIION  MK)I|T  NUM»tR(si 

JB)SR.I|1.  95-0432 


u  MAMI  riMOIIMiNO  ib  On<i  SYMIOi  7$  «tAMi  Of  MONlTQftlNC  QltOAMlZATON 

Hatfield  Marine  Science  Centei 
Oreqon  State  University _ Air  Force  Office  of  Scientific  Research/NL 


•c  AOOMSS  fcny.  »«».  tna  nrco*) 
Narine  Science  Drive 
Newport,  OR  97365 


7b  AOORfSSKNjr.  SUtt.  «<W  /TCo*) 

Building  410 
Bolling  AFB,  DC  20332 


U  KAMI  Of  fUNCMNG /SeONSORlNG 
OaCANOATlON 

AFOSR  _ 


Sc  AOOACSS  (Citr.  5t«t«,  «nd  /IT  Codti 

Building  410 

Bolling  AFB.  DC  20332-6448 


t1  TfTLf  pndMV  Svcwnty  GmaAcrowU  n 


lb  OfFKf  SYMIOI  t  MOCURCMtNT  MSTRUMtNT  lOCNTifiCATlON  NUMItR 
Of  «0pNc«aati  ^ 

NL  V  H^ioQO  -  qs  'Cr  -O  WD 


10  SOURCf  Of  fUNOtNC  NUMICRS 


MOGRAM  RROJCa  TASK 

CUMCNT  NO  NO.  NO 

6n02F  2312 


'  Parallel  Processing  and  Learning:  Variability  and  Chaos  in 

Self-Organization  of  Activity  in  Groups  of  Neurons 


12.  eeRSONAL  AUTHORS) 

George 


is«.  rrec  of  rcrort  lisb.  time  covered 

Fina.l  Reoort _ |  from  92/1 /1 5  t 


IS.  SOeeUMiNTARY  notation 


'mifisil 


14.  DATE  OF  REFORT  (Yt4f.  MontR.  OtW 

14  _ 


COSATI  COOES 


GROue  SUt-CROUF 


II  SUUECT  TERMS  (ConcmiM  on  fvvtfit  0  noctowy  ontf  tdtnofy  by  bhtk  number) 

Parallel  processing,  learning,  catalytic  networks, 
attractors,  gradient  descent 


If.  AasTRACT (Cononoo on w nwwo' 4nd iSwcNy ay a<oc* nombw) Abstract  is  Continued  on  reverse  side. 

GOALS:  Motivated  partly  by  our  previous  work,  we  have  attempted: 

(1)  To  determine  whether  there  is  a  global  mechanism  that  automatically  adjusts  the 
strengths  of  the  many  connections  that  take  place  between  neurons. 

(2)  To  determine  the  source  of  variations  in  neuronal  firing  observed  during 
behaviorally  meaningful  neural  activity,  whether  the  variations  arise  from  deterministic 
processes  or  nondetermimstic  ones,  and  to  identify  the  S5maptic  or  membrane  mechanisms  that 
may  give  rise  to  them. 

Because  biological  systems  are  difficult  to  control,  we  have  used  computer  simulations 
to  examine  these  problems.  The  results  indicate  that  the  findings  may  be  addressable  in 
biological  systems,  particularly  in  cell  cultures  of  two  or  three  selectively  connected  neurons. 
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19.  Abstract,  continued. 

RESULTS:  Five  illustrations  are  used  in  this  report  to  discuss  the  major  features  of  the  results. 

Figures  1  and  2  are  directed  to  the  first  of  the  above  problems.  A  conjecture  is  outlined 
and  evaluated:  If  a  group  of  neurons  can  sustain  recurring  activity  whose  d)mamics  represents 
an  attractor,  and  if  the  synaptic  strengths  and  thresholds  are  set  by  activity-dependent 
mechanisms,  the  dissipative  properties  of  the  attractor  constrain  the  flow  of  activity  among 
these  neurons  in  such  a  way  that  all  synapses  and  thresholds  are  optimally  set  with  respect  to 
one  another. 

Appendix-1  provides  an  extended  discussion  of  this  conjecture,  and  of  other  findings 
that  emerge  from  it,  and  places  it  within  the  context  of  findings  in  the  literature.  Appendix-1  is 
presently  under  review  in  Behavioral  and  Brain  Sciences. 

Figures  3-5  are  directed  to  the  second  of  the  above  problems.  We  used  the  rationale  that 
one  S5maptic  input  acts  as  a  perturbation  stimulus  to  another;  i.e.,  a  given  synaptic  input  that 
leads  to  the  generation  of  an  action  potential  acts  on  membrane  impedance  that  has  been 
preconditioned  by  some  preceding  event.  In  the  present  series  of  experiments,  we  used  a  weak 
preconditioning  delta-function  current  impulse  as  the  conditioning  stimulus  to  determine  its 
effect  on  the  latency  of  a  synaptically  activated  action  potential  (spike).  The  findings  show 
clearly  that  the  membrane  is  highly  sensitive  to  the  temporal  pattern  of  the  input  signal.  The 
output  spike  train,  however,  has  a  temporal  structure  that  is  significantly  altered  by  comparison 
to  the  input.  More  importantly,  the  information  flow  across  the  S5mapse  is  highly  degenerate; 
i.e.,  that  degenerate  conditions  may  emerge  naturally  from  a  completely  deterministic  system.  It 
is  expected  that  our  future  work  will  show  that  variations,  and  other  dynamical  characteristics 
of  network  activity,  naturally  emerge  from  such  conditions. 

Appendices  2  &  3  discuss  these  findings  in  detail,  and  place  them  within  the  context  of 
the  literature  over  the  past  30  years  that  has  dealt  with  the  possibility  that  neurons  may  be 
sensitive  to  the  pattern  of  the  input  signals  that  they  receive.  Appendices  2  &  3  are  near  final 
versions  of  papers  that  will  be  submitted  for  publication  in  Biological  Cybernetics. 

Publication  that  have  appeared  in  print  over  the  past  three  years: 

Andrade  MA,  Nuno  JC,  Moran  F,  Montero  F,  Mpitsos  GJ  (1993)  Complex  dynamics  of  a  catalytic  network 
having  faulty  replication  into  an  error  species.  Phys  D  63:21-40 
Burton  RM,  Mpitsos  GJ  (1992)  Event-dependent  control  of  noise  enhances  learning  in  neural  networks. 
Neural  Networks  5:627-637 

Mpitsos  GJ,  Burton  RM  (1992)  Convergence  and  divergence  in  neural  networks:  Processing  of  chaos  and 
biological  analogy.  Neural  Networks  5:605-625 

Mpitsos  GJ,  Soinila  S  (1993)  In  search  of  a  unifying  theory  of  biological  organization:  What  does  the  motor 
system  of  a  sea  slug  tell  us  about  human  motor  integration?  In:  Newell  KM,  and  Corcos  D,  (eds) 
Variability  and  Motor  Control,  Human  Kinetics,  Champaign,  pp  225-290 
Soinila  S,  Mpitsos  GJ,  Soinila  J  (1992)  Enkephalin  immunohistochemistry:  Model  studies  on  conjugation 
reaction  and  fixation.  J  Histochem  Cytochem  40:231-239 

Appendices: 

1.  Mpitsos  GJ  (1995)  Attractor  gradients:  architects  of  developmental  organization.  Under  review  in 

Behavioral  and  Brain  Sciences 

2.  Edstrom  JL,  Mpitsos  GJ  (1995)  Predicting  the  influence  of  perturbing  currents  on  spike  latencies: 

Determinism  and  degeneracy  of  information  in  synaptic  transmission.  A  revised  version  will  be 
sent  to  Biological  Cybernetics. 

3.  Edstrom  JL,  Mpitsos  GJ  (1995)  Simple  neuronal  membrane  is  sensitive  to  the  temporal  structure  of 

input  signals.  A  revised  version  will  be  sent  to  Biological  Cybernetics. 
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L  FINAL  REPORT 


1  nissipative  action  of  network  firing  patterns  sets  all  svnapses  QPtimaUv  With  reSPect  \Q  OJIS 
another 

The  following  conjecture  is  examined: 

(a)  Assume  that  the  network  is  initially  randomly  connected,  and  that  the  neurons  are  firing 
action  potentials. 

(b)  Somewhere  in  the  network,  two  or  three  neurons  may  have  firing  patterns  that  are 
sufficiently  correlated  with  one  another  that  their  synapses  xmdergo  activity-dependent 
changes.  If  dissipative  action  begins  to  emerge,  the  synaptic  weights  in  the  developing 
network  must  have  some  interrelated  range  of  values,  else  the  attractor  would  not  exist. 

(c)  The  emerging  attractor  constrains  the  flow  of  activity,  causing  further  activity-dependent 
changes  and  refinement  of  the  attractor  spatio-temporally. 

(d)  Neurons  that  converge  onto  the  same  follower  neurons  may  compete  (Merzenich,  et  al., 
1978;  Merzenich,  et  al.,  1983a;  Merzenich,  et  al.,  1983b;  Merzenich,  et  al.,  1984;  Mpitsos,  et  al., 
1978;  Recanzone  and  Merzenich,  1992)  for  occupancy  of  the  synaptic  space  available  on  the 
follower  neuron  such  that  connections  may  be  made  or  broken,  structurally  molding  the 
neuroarchitecture 


Rphavior  of  a  fixed  network:  To  illustrate  this  conjecture,  first  consider  a  network  that 
has  already  been  set  and  has  the  strengths  of  all  of  its  synapses  frozen.  Figure  1C  shows  the 
network.  For  the  present  discussion,  we  may  exclude  Cell4;  its  use  is  discussed  in  Appendix-1 
where  some  of  the  details  of  the  network  are  also  given  (detailed  descriptions  of  the  membrane 
and  synapse  characteristics  are  given  in  Appendbc-2).  The  three  cells  are  connected  by  excitatory 
(synapses  shown  by  open-circle  terminals)  and  inhibitory  connections  (sjmapses  shown  by 
filled-cirdes).  The  delay  between  the  synapses  was  set  at  3  msec.  Simulation  of  electrogenic 
activity  used  the  Hodgkin-Huxley  model  of  the  squid  giant  axon  (future  research  will  use  more 
biologically  complex  neurons).  The  strength  of  the  synapses  is  indicted  by  ti\e  thickness  of  the 
connection  "axons"  and  numerically  in  milliSiemens.  Steady  depolarizing  cvirrent  was  injected 
into  Cell-1,  causing  it  to  fire  action  potentials  which  activate  the  remaining  <»lls  in  the 
network. 

Figiu-e  1 A  shows  a  3-cycle  firing  pattern  in  Cell2.  Starting  from  the  simulation  time  at 
680  msec,  seven  intervals  between  the  action  potentials  are  shown,  where  the  3^cle  repetition 
sequence  consists  of  intervals  having  15.4  msec,  35.0  msec,  and  46.3  msec  durations.  For  the 
pmposes  of  the  illustration  only  two  3-cycle  periods  are  shown,  but  many  more  3-cycle 
sequences  preceded  them.  This  particular  firing  pattern  was  obtained  by  driving  the  network 
using  steady  depolarization  of  Celll  and  selectively  adjusting  the  strength  of  the  inhibitory 
synapse  that  Cell3  makes  on  Cell2  tmtil  the  3-cycle  was  obtained. 

Certain  other  combinations  of  synaptic  weights  also  give  similar  3-cycle  activity,  just  as 
Gardner  (1990)  has  demonstrated  that  different  synaptic  streng^  between  four  identified 
neurons  in  the  sea  slug  Aplysia  generate  similar  network  function.  The  dashed  line  below  the 
action  potential  trace  shows  the  time  dming  which  the  network  was  perturbed  by  injecting 
Cell2  with  a  125  msec  electrical  current  pulse  that  pushed  the  membrane  potential  below  the 
threshold  level  for  the  genesis  of  action  potentials,  except  for  one  action  potential  (labeled  by  a 
"1")  arising  from  strong  activity  in  one  of  the  other  cells  in  the  network.  After  the  cessation  of 
the  applied  current,  the  cell  rebovmded  with  a  rapid  burst  of  three  action  potentials  (labeled  "2", 

"3",  "4"),  and  then  recovered  to  its  original  firing  pattern.  _ _ 

The  pattern  of  intervak  shown  in  Fig.  1 A  may  be  plotted  in  a  retira  imp  in  which  one  Qn£_^ 
interval  in  the  sequence  is  plotted  on  the  horizontal  axis  and  the  succeeding  interval  is  plotted 
on  the  vertical  axis.  One  himdred  intervals  were  used  to  construct  the  3-cycle  of  the 
unperturbed  activity;  all  fell  sequentially  on  the  three  filled  circles  in  Fig.  IB.  The  return-map 
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positions  of  the  intervals  during  and  after  the  application  of  the  125  msec  perturbation  pulse  are 
shown  by  the  niunbered  and  open  triangles.  The  identifying  numbers  in  the  return  map  are  the  I 
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same  as  those  in  the  trace  shown  in  Fig.  lA,  where  the  number  over  a  given  action  potential 
indicates  the  interval  between  that  action  potential  and  the  preceding  one. 

Behavior  of  a  changeable  network:  The  schematic  in  Fig.  2  shows  how  the  network 
examined  previously  may  be  altered  following  a  perturbation  stimulus.  The  original  network  is 
shown  in  Fig.  2A.  It  is  the  same  as  the  one  shown  in  Fig.  1,  as  are  the  activation  conditions,  but 
the  strengths  of  the  synapses  are  now  made  available  for  activity-dependent  changes.  If  Cell2  is 
depolarized  for  an  arbitrary  length  of  time,  it  begins  to  generate  action  potentials  initially  at  a 
higher  frequency  than  before,  but  soon  the  balance  of  excitatory  and  inhibitory  feedback 
readjusts  its  rate  of  firing.  The  activity-dependent  changes  in  a  synapse  depend  on  the  temporal 
correlations  of  activity  in  the  presynaptic  and  postsynaptic  cells  which  will  vary  over  time.  The 
d)mamics  giving  rise  to  these  correlations  is  extremely  complicated  and  time  dependent,  but 
Fig.  2B  has  caught  the  changed  s)mapses  at  a  stage  (shown  by  the  numbers  next  to  the  axons) 
that  cause  Cell2  to  generate  a  6-cycle  firing  pattern.  The  open  triangles  in  the  return  map  of  Fig. 
2B  show  the  values  of  the  six  intervals  between  the  action  potentials.  The  filled  circles  show  the 
original  3-cycle. 

When  Cell2  is  released  from  the  depolarizing  current  (Celll  is  under  continuous 
depolarization  to  keep  the  network  active),  its  firing  pattern  may  remain  in  the  6-cycle  state,  but 
if  the  attractor  is  not  sufficiently  well  formed,  genesis  of  action  potentials  in  the  three  neurons 
may  drift  to  some  other  set  of  patterns,  or  it  may  eventually  settle  back  to  the  original  3-cycle 
state  (and  perhaps  with  a  different  set  of  synaptic  strengths  between  the  neurons). 

The  aim  here  is  not  to  say  that  the  6-cycle  per  se  is  formed,  nor  that  the  original  3-cycle  is 
regenerated  in  the  recovery,  but  that  some  attractor  is  generated,  and  that  its  ^adients  are  what 
adjusts  or  globally  "directs"  the  local  activity-dependent  synaptic  changes.  This  is  different  than 
would  be  imagined  were  we  to  view  networks  as  switchboards  or  reflexes  by  which  change  is 
based  only  on  local  activity-dependent  effects.  From  such  effects  it  might  be  postulated,  for 
example,  that  associative  learning  in  whole  animals  is  based  on  changes  in  individual  synapses; 
reviewed  in.  By  the  conjectured  gradient  method,  however,  it  is  obvious  that  changing  one 
synapse  causes  changes  in  all  of  them.  Local  rules  are  a  work,  but  they  are  guided  by  a  more 
global  principle. 

Appendix-1  places  this  in  the  context  of  findings  in  artificial  neural  networks,  and 
discusses  how  dissipative  mechanisms  may  lead  to  a  number  of  effects,  such  as  the  limitations 
of  the  functional  size  of  the  network  and  the  t)q)es  of  information  that  may  be  stored  among  the 
coimections  in  the  network. 

2.  Sensitivity  of  a  neuron  to  the  pattern  of  input  stimuli.  &  the  emergence  of  degeneracy  in  the 
information  flow  across  a  simple  synapse 

How  much  of  the  input  stimulus  pattern  does  the  output  firing  pattern  of  a  neuron 
represent?  How  does  the  membrane  transform  the  input  signal?  What  are  the  filter  properties  of 
neuron  membrane?  Does  the  membrane  of  posts5maptic  neurons  act  as  a  filter  that  is  sensitive 
to  the  pattern  of  the  input  data  stream  arriving  from  presynaptic  neurons?  Being  able  to 
describe  the  membrane  as  a  linear  filter  in  order  to  understand  the  answers  to  such  questions 
has  been  a  subject  of  interest  for  many  decades,  as  discussed  in  Appendix-2.  Moreover,  the 
possibility  that  neurons  may  be  sensitive  to  the  temporal  patterns  of  input  signals  has  important 
implications  in  our  understanding  of  neurointegrative  mechanisms  (Softky,  1995). 

We  have  devised  experiments  that  address  such  questions  in  computer  simulations.  The 
work  described  in  Appendices  2  &  3  uses  simple  synapses  and  neuron  membrane,  as  may  be 
found  in  the  giant  synapse  and  axon  of  the  squid.  Subsequent  studies  will  extend  the  work  to 
more  complex  neurons  and  synapses.  The  results  give  some  insight  into  the  above  questions, 
and  also  provide  a  basis  for  understanding  how  at  least  one  source  of  nondeterministic 
variation  in  the  firing  patterns  of  neurons  may  emerge  naturally  from  a  deterministic  process. 
The  rationale  for  starting  with  studies  that  examine  the  membrane  within  its  linear  response 
range  is  also  discussed  in  the  Introduction  and  Discussion  of  Appendix-2. 

The  results  in  Fig.  3  provide  the  foundation  for  the  work  covered  in  Appendices  2  &  3, 
and  for  much  of  our  work  that  we  shall  be  doing  over  the  next  several  years.  The  design  of  the 
experiments  used  to  obtain  the  results  is  a  simple  perturbation  method.  The  family  of  curves 
shown  in  Fig.  3  C  &D  are  the  composite  of  many  40  msec  simulation  sweeps.  In  each  sweep,  a 


single  excitatory  postsynaptic  current  (EPSC)  was  activated  at  25  msec  .into  the  sweep.  This 
produced  a  constant-latency  action  potential  (Fig.  3B)  when  the  postsynaptic  neuron  was  not 
presented  the  conditioning,  perturbation  stimuli.  The  effects  of  perturbatiohs  are  shown  in  Fig. 
3  C  &  D ,  and  are  compared  against  this  standard  response  as  a  changes  in  die  spike  latency.  A 
single  excitatory  (solid-line  traces)  or  inhibitory  (dotted-line  traces)  current  impulse,  lasting  one 
integration  step,  was  presented  during  the  first  30  msec  of  each  simulation  sweep.  In  Fig.  3  C  & 
D,  the  position  of  the  impulse  was  moved  pro^essively  from  left  to  right  in  the  simulation 
sweep.  Each  curve  was  generated  using  one  impulse  amplitude.  Progressing  outwardly  from 
the  zero-latency  change,  the  amplitudes  in  multiples  of  lO'lO  Amp  were  (±)  2, 4, 6, 8, 10. 

The  most  important  finding  in  Fig.  3  is  that  a  given  latency  can  be  produced  by  many 
different  temporal  positions  of  the  impulses.  A  horizontal  line  drawn  in  Fig.  3  to  represent  a 
given  latency  intersects  not  only  one  ciurve  in  two  places,  but  many  other  curves  as  well:  i.e., 

§\e  input/output  function  of  synapses  imder  perturbation  produce  a  many-to-one  mapping.  An 
observer  of  the  output-spike  latencies  is  ignorant  of  the  temporal  positions  of  the  stimuU  that 
produced  the  latency  changes  because  a  given  latency  can  not  be  used  to  uniquely  identify  the 
timing  of  the  perturbation  impulse  that  generated  it. 

This  raises  many  important  research  questions  relating  to  strategies  that  biological 
systems  may  have  taken  to  accommodate  to  such  degeneracies.  One  possibility  is  that  the 
degeneracy  may  be  useful  in  generating  response  variations  that  may  be  useful  in  controlling 
how  networks  settle  into  or  exit  response  patterns.  The  relationship  of  these  input/ output 
functions  to  studies  in  biological  systems  of  the  impulse  response  fimction  (^),  the  average 
current  trajectory  (ACT),  and  membrane  impedance  is  discussed  in  Appendix-2. 

3.  Sensitivity  of  simple  nevu-on  membrane  to  the  pattern  of  input  signals 

(a)  Random  stimulation  over  the  temporal  range  of  input/ output  functions  showg  that 
thP  mpmbrane  acts  as  a  nonrandom  Hlter.  The  experiments  shown  in  Fig.  4  are  the  same  as 
those  in  Fig.  3  except  that  the  timing  of  the  perturbing  impulses  was  randomly  controlled. 
100,000  simulation  sweeps  were  run.  The  histogram  in  Fig.  4B  shows  that  the  timing  of  the 
impulses  covered  the  7-30  msec  interval  of  the  simulations  evenly.  The  input/ output  function 
in  Fig.  2  is  rotated  vertically  by  comparison  to  the  curves  in  Fig.  3  so  that  the  time-axis 
extends  upward  and  the  latency-axis  extends  horizontally  from  left  to  right.  Only  one 
input/output  curve  is  shown  because  all  of  the  impulses  had  the  same  4xl0'10Amp 
amplitude.  Figure  4C  shows  that  the  histogram  of  the  output  latencies  is  significantly  distorted 
by  comparison  to  the  distribution  of  the  impulses  that  produced  the  latency  changes,  showing 
that  membrane  of  the  postsynaptic  cell  imposes  structure  on  the  signals  that  it  receives.  How 
this  occurs  is  shovm  more  explicitly  below. 

(b)  Stimulation  of  different  temporal  regions  within  the  range  of  the  inPUt/ootput 
functions  shows  that  the  membrane  filter  imposes  structure  on  its  input  signals  and  that  the 
output  is  morp  complex  than  flte  input  stimulus  pattern,.  These  experiments  are  the  same  as 
those  shown  in  Fig.  4  except  that  the  temporal  position  of  the  perturbing  impulses  was 
controlled  by  the  recursive  chaotic  logistic, /(x)  =  3.7(l-x)x,  where  x  is  in  the  unit  interval;  the 
output  of  this  function  was  scaled  so  that  it  covered  different  regions  of  the  simulation  sweep. 
Return-map  plots  of  this  quadratic  function,  in  which  one  value  f(x)  is  plotted  against  the 
preceding  one,  produce  a  typical  inverted-U  map  a  quadratic  fimction  tiiat  can  be  compared  to 
the  return  maps  of  the  spike  latencies. 

The  effect  of  changing  the  position  and  range  of  the  impulses  is  shown  in  Fig.  5.  T^e 
presentation  times  of  seven  different  sets  of  input  stimulus  trams  are  shown  by  the  filled  circles; 
tiiey  are  shifted  vertically  with  respect  to  one  another  in  order  to  illustrate  their  positions. 
Figure  5B  shows  the  return  maps  of  the  spike  latencies;  each  imp  is  identified  numerically  by 
the  temporal  range  of  the  input  stimulus  times  that  generated  it.  The  return  maps  of  the 
latencies  are  significantly  distorted  by  and  non-quadratic;  i.e.,  whereas  the  input  of  the 
quadratic  logistic  is  a  simple  second-order  polynomial,  the  multi-looped  maps  of  the  latency 
return  maps  require  four^  or  higher-order  polynomials  to  define  them. 

Additionally,  the  ends  of  the  spike-latency  return-maps  may  cross.  This  occurs 
whenever  the  stimulus  pattern  sufficiently  spans  a  humped  (vertically  convex  or  concave) 
region  of  the  i/o  functions.  An  example  is  shown  in  Fig.  5B  by  the  map  labeled  "14-19."  These 
crossed-loop  maps  are  degeneracies  that  cannot  exist  in  an  deterministic  system  since  the  flow 
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of  events  at  the  crossing  point  is  not  uniquely  determined.  However,  this  can  be  resolved  by 
plotting  the  return  maps  in  three  dimensions  (not  shown)  where  one  latency  (X)  is  mapped  on 
the  x-axis,  the  next  interval  (Xn+l)  is  on  the  y-axis,  and  the  third  interval  (Xn+2)  in  succession  if 
mapped  on  the  z-axis.  Rotating  this  3-dimensional  image  shows  that  the  maps  do  not  actually 
cross,  but  that  they  only  appeared  to  do  so  because  the  2-dimensional  return  map  is  a 
projection  of  the  3-dimensional  one.  Dynamically,  the  input  data  stream  is  a  1-dimensional 
process,  whose  dynamics  can  be  characterized  in  two-dimensional  return-map  embedding.  The 
output  data  stream  is  also  a  1-dimensional  process,  but  it  requires  at  least  three-dimensional 
embedding  to  resolve.  Thus,  the  filter  properties  of  the  postsynaptic  neuron  produces  an  output 
firing  pattern  whose  temporal  characteristic  are  not  only  a  distortion  of  the  dynamics  in  the 
input  signal,  they  also  produce  an  output  whose  dynamics  live  an  embedding  space  having  a 
higher  dimension  than  the  dynamics  of  the  input  data  stream. 

SIGNIHCANCE: 

These  findings  raise  a  number  of  serious  questions  about  how  information  is  transferred 
across  synapses,  and  how  neurons  or  circuits  of  neurons  may  have  accommodated  to  the 
complexities  produced  by  the  input/output  functions.  It  is  conceivable  that  the  variations  may 
play  a  beneficial  role  in  neurointegrative  processes.  It  is  particularly  interesting  that  a 
completely  deterministic  process  may  generate  non-deterministic  responses.  Unlike  chaos,  in 
which  the  variations  themselves  are  deterministic,  the  degeneracies  indicated  by  the  findings  in 
Fig.  4  &  5  may  lead  to  nondeterministic  variations.  The  findings  shown  in  Figs.  1  &  2  provide  a 
rationale  with  which  to  conceive  of  how  complex  neuropil  may  be  formed,  and  attractors 
provide  a  functional  way  to  imderstand  how  information  distributed  widely  in  a  network.  We 
believe  that  many  integrative  functions,  that  may  not  be  follow  easily  from  traditional  biological 
studies,  naturally  emerge  from  the  findings  outlined  above.  Fuhire  work  will  inquire  into  these 
possibilities. 
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HGURE  LEGENDS 


FIGURE  1.  Dissipative  action  in  the  simulation  of  small  biologically  realistic  networks.  The  network 
(C)  was  activated  by  steady,  low-level  depolarizing  current  applied  to  Celll.  A:  Time  series  of 
action  potentials  obtained  from  Cell2.  The  activity  consists  of  a  3-cycle  in  which  three  intervals 
between  action  potentials  repeatedly  appear,  as  shown  by  the  first  seven  action  potentials.  A 125 
msec  hyperpolarizing  perturbation  pulse  was  presented  to  Cell2  (marked  by  the  dashed  Ime).  One 
action  potential  escaped  during  the  hyperpolarization.  After  the  release  of  the  hyperpolarizing 
pulse,  Cell2  reboimded,  generating  three  action  potentials  in  rapid  succession,  and  then  relaxed 
toward  the  original  3-cycle  intervals.  B:  Shows  the  return  map  of  the  original  3-cycle  (filled  circles) 
and  the  post-perturbation  recovery  (open  triangles).  The  numbers  in  (B)  ^e  the  sanie  as  in  A,  and 
represent  the  interval  between  the  numbered  action  potential  and  the  action  potential  preceding  it. 
C:  Structure  of  the  network;  the  thickness  of  the  connections  are  proportional  to  the  maximal 
synaptic  strengths;  the  numbers  show  the  strengths  in  milliSiemens.  Excitatory  synapses  (open 
circles)  have  a  reversal  potential  of  45  mV.  Inhibitory  synapses  (filled  circles)  have  a  reversal 
potential  of  -82  mV.  Open  and  close  time  constants  for  both  types  of  synapses  are  3  and  20  msec, 
respectively.  Transmission  delays  were  3  msec.  The  simulations  were  run  GENESIS  obtained  from 
the  California  Institute  of  Technology,  Pasadena,  and  used  squid  axon  membrane  (Hodgkin  & 
Huxley,  1952). 

FIGURE  2.  Attractors  may  adjust  synaptic  strengths  globally.  A:  The  same  network  that  produced 
the  3-cycle  activity  shown  in  Fig.  1  is  used  here.  B:  Activity-dependent  emergence  of  new 
cormection  strengths:  The  network  was  activated  using  Celll  to  produce  the  3-cycle.  After  start-up 
transients  were  dissipated,  Cell2  was  depolarized  tonically  while  maintaining  the  depolarization 
applied  to  Celll.  AH  synapses  in  the  network  were  allowed  to  change  using  a  variant  of  Hebb’s 
rule.  The  new  synaptic  strengths  are  indicated  by  Hie  thickness  of  the  connection  lines  between  the 
neurons  and  by  the  numerical  values  placed  next  to  the  lines  (compare  A  and  B).  C:  A  return  map 
showing  that  the  new  connection  strengths  generate  a  6-cycle  pattern  (open  circles)  in  the  firing  of 
Cell2;  the  original  3-cycle  is  shown  by  (he  filled  symbols.  When  shorter  or  weaker  stimuli  are 
presented  to  Cell2,  the  s)maptic  strengths  do  not  settle  sufficiently  into  the  values  required  to 
generate  the  6-cycle.  The  synaptic  strengths  may  fluctuate  as  the  interrelated  firing  patterns  of  the 
cells  in  the  network  seek  one  gradient  or  another.  The  temporal  structure  of  these  patterns  may 
drift  imtil  a  strong  enough  gradient  emerges  which  then  forces  the  interrelated  sfiengths  of  the 
synapse  into  some  more  stable  set.  The  activity  dependent  changes  in  the  synaptic  strengths, 
discussed  a  little  later,  used  Mike  Vanier's  implementation  of  Hebbian  s5mapses  in  GENESIS. 


FIGURE  3;  The  timing  of  single  preconditioning  current  impulses  produces  complex  Ranges  in 
action  potential  latencies.  A  single  current  impulse  was  given  in  each  40  msec  simulation  sweep;  the 
timing  was  allowed  to  overlap  the  EPSC.  The  design  of  the  experiment  was  aimed  at  imderstanding 
how  the  temporal  position  of  the  impulse  affected  the  latency  of  the  spike  produced  by  the  EPSC.  A 
series  of  such  experiments  defined  input/output  functions  that  relate  impulse  timing  to  spike  latency. 
A:  Onset  (25  msec)  and  time  course  of  the  EPSC.  B:  Action  potential  produc^  by  the  EPSC  in  the 
absence  of  a  preconditioning  current  pulse.  C  &  D:  latencies  of  action  potentials  produced  in 
hyperpolarized  (-9.5  x  lO'H  Amp)  and  normal  membrane  in  response  to  depolarizing  (solid  ^rves) 
and  hyperpolarizing  (dotted)  current  pulses.  Each  curve  was  obtained  using  a  different  amplitude  of 
the  current  pulse.  The  incomplete  lines  in  some  of  the  negative  current  isopleths  indicate  suppression 
of  the  EPSC-evoked  action  potential  at  those  current/ time  combinations.  The  interval  between  the 
points  on  the  cvirves  is  250  msec.  The  amplitude  for  the  five  0.010  ms  depolarizing  and 
hyperpolarizing  current  pulses  was,  in  multiples  of  lO'^^  Amp:  (±)  2, 4, 6, 8, 10.  Ggyn  =  3  x  10"^ 
Siemens  at  the  normal  resting  potential  and  4.0536  x  10'8  Siemens  at  the  hyperpolarized  potential,  a 
value  chosen  by  trial  and  error  to  give  the  same  unperturbed  action  potential  latency  as  the  normal 
control  EPSC.  Integration  time  step  =  0.010  ms. 


FIGURE  4:  Recent  history  of  impedance  changes  that  occiu-  imder  random  perturbation 
generates  nonrandom  firing  of  EPSC-evoked  action  potentials.  A:  Relation  of  impulse 
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presentation  time  (vertical  axis)  to  action  potential  latency  (horizontal  axis).  B:  Density  of 
randomly  generated  impulse  presentation  times;  vertical  axis  is  the  same  as  in  (A),  A  single 
randomly  timed  current  impulse  (4  x  lO'^O  Amps,  lasting  one  .010  ms  integration  step)  was 
presented  in  the  7-30  ms  interval  of  each  35  ms  simulation  sweep.  A  suprathreshold  EPSC  was 
initiated  25  ms  into  the  simulation.  C:  Histogram  of  the  EPSC-evoked  action  potentials  the 
resulted  from  100,000  simulation  sweeps. 

FIGURE  5:  The  temporal  position  along  the  latency  input/output  functions  represent  different 
distortions  of  the  structure  of  the  input  function  generating  the  perturbation  pulses.  A:  The  tiiin 
line  shows  the  action  potential  latencies  arising  from  a  composite  of  logistic-activated  sequence 
of  current  pulses  having  constant  amplitude  (9  x  lO'l®  Amp;  .010  ms).  Seven  of  these  sequences 
are  shown  by  the  dotted  tmrves  which  have  been  shifted  vertically  to  identify  them.  The 
irregular  spacing  between  the  dots  in  each  line  arises  from  the  logistic  scaling  function  T 
described  in  the  text.  B:  Rehun  maps  of  the  intervals  in  six  of  the  sequences  in  (A).  The  numbers 
identify  the  approximate  range  of  times  spanned  by  each  sequence  along  die  input/ output 
function. 
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SHORT  ABSTRACT 

Biological  systems  have  many  interacting  components  that  together  produce 
coherent  actions  under  conditions  in  which  each  component  has  relatively 
limited  information.  How  coherence  emerges  in  such  complex  systems  is 
presently  unclear.  Using  findings  from  biological,  computational  neural 
network,  and  dynamical  system  studies,  I  discuss  how  system-wide  attractor 
gradients  may  provide  a  global  organizing  principle  that:  adjusts  the  local 
interactions  optimally  with  respect  to  one  another;  limits  network  size;  sculpts 
the  neuroarchitecture,  further  redefining  the  gradients  themselves;  and 
imposes  new  types  of  stored  information.  Problems  encountered  in  defining 
the  existence  of  attractors  are  discussed. 


ABSTRACT 

All  biological  systems  are  composed  of  many  parts  that  must  work  together  to 
produce  coherent  adaptive  responses.  The  many  nonlinear  intercormections 
between  neurons  in  an  assembly  or  between  individuals  in  any  population  pose 
analogous  questions  to  analytical  studies,  e.g:  How  does  one  handle  the  many 
degrees  of  freedom  to  know  how  the  system  as  a  whole  functions?  How  does 
global  function  emerge  from  the  action  of  individuals  whose  decisions  are  based 
on  local  information?  Many  simplifications  may  occur  if  the  system  is  dissipative; 
i.e.,  if  it  has  an  attractor  such  that  it  tends  to  return  to  a  characteristic  state  in 
response  to  external  perturbation.  I  discuss  possibilities  emerging  from  studies  in 
neural  networks  and  dynamical  systems  to  examine  how:  (a)  Spatio-temporal 
attractor  basins  act  as  a  global  force  on  populations  of  neurons,  restricting  the 
correlations  between  their  activities,  (b)  If  the  strengths  of  the  connections  are 
activity-dependent,  gradients  set  the  connection  strengths  between  all  neurons, 
and  the  response  thresholds  of  individual  neurons,  optimally  with  one  another,  (c) 
limit  network  size,  and  (d)  control  the  types  of  information  that  a  system  stores. 
The  findings  may  provide  insight  into  mechanisms  that  might  not  have  been 
considered  from  classical  approaches  that  view  brain  function  as  arising  from 
simple  reflexes  and  "switchboard"  circuitry.  Similar  effects  may  occur  in  many 
different  systems,  from  cellular  to  societal  ones,  that  generate  attractors  and  in 
which  the  response  thresholds  and  strengths  of  interactions  between  the 
component  parts  are  adjusted  by  activity-dependent  mechanisms. 

Key  words:  Self-organization,  neural  networks,  multifunctional  systems, 
activity-dependent  changes,  gradient  systems,  thresholds,  central  pattern 
generators,  neural  development,  learning,  population  dynamics. 


INTRODUCTION 

Common  conceptual  and  experimental  problems.  All  biological 
systems,  whether  cellular  ones,  organismal  ones,  or  ones  involving 
populations  of  organisms,  have  the  common  feature  of  being  composed  of 
many  parts  that  must  work  together  to  produce  coherent  responses  or 
behaviors.  Because  of  these  attributes,  similar  conceptual  and  experimental 
problems  arise  in  widely  disparate  systems  such  that  studies  in  the  social 
sciences  are  faced  with  many  of  the  same  difficulties  as  those  in  the 
neurosciences.  For  example,  individual  persons  or  neurons  communicate  with 
many  others  within  the  potentially  coactive  populations,  yet  must  generate 
responses  based  largely  on  local  rules  because  no  single  individual  has 
information  about  the  population  as  a  whole  or  about  the  context  within 
which  the  population  must  act.  The  present  paper  deals  with  the  organization 
of  neural  tissues,  but  the  findings  that  are  discussed  may  also  extend 
heuristically  to  other  systems.  The  problem  will  be  examined  of  how  a 
globally  acting  principle  may  emerge  from  local  interactions  in  the  system 
and,  if  the  coimections  are  changeable,  how  it  affects  the  connections 
themselves. 

I  shall  discuss  that  attractors  may  provide  one  such  principle. 

Attractors  are  an  old  idea  (Abraham  &  Shaw,  1983),  and  have  long  been 
thought  to  be  useful  in  developing  a  biological  and  computational  theory 
relating  to  disparate  systems,  from  chemical  ones,  to  cellular  ones,  to  ones 
composed  of  populations  of  animals  and  possibly  even  to  evolution 
(Waddington,  1968-1972).  There  are  many  sources  dealing  with  the  use  of 
attractors  and  the  qualitative  shift  m  dynamics  or  bifurcations  in  forced  and 
unforced  systems,  e.g.,  (Andrade,  et  al.,  1993;  Chay  &  Rinzel,  1985;  Cohen  & 
Grossberg,  1983;  Eigen  &  Schuster,  1979;  Farmer,  et  al.,  1987;  Farmer,  et  al., 
1986;  Guckenheimer  &  Holmes,  1983;  Haken,  1983;  Hirsch  &  Smale,  1974; 
Hofbauer  &  Sigmtmd,  1988;  Holden,  1985;  Hopfield,  1982;  Kiippers,  1983;  Le 
Cun,  et  al.,  1991;  Schnabl,  et  al.,  1991;  Skarda  &  Freeman,  1987;  Ueda,  1992; 
Widrow  &  Steams,  1985).  Additionally,  a  large  literature  has  emerged,  based 
on  the  seminal  work  of  (Grassberger  &  Procaccia,  1983;  Packard,  et  al.,  1980; 
Takens,  1981)  aimed  at  computing  attractor  dimensions  and  the  minimal 
number  of  degrees  of  freedom  or  dimensions  required  to  define  attractors 
from  experimental  observations.  A  recent  review  by  Abarbanel  et  al.  (1993)  is 
a  useful  accoimt  of  some  of  the  methods  and  directions  for  future  work.  A 
critical  account  of  a  number  of  these  methods,  methods  for  controlling  the 
dynamics  of  a  system,  methods  for  determining  whether  the  responses  of  an 
experimental  system  are  deterministic,  and  software  for  making  such 
assessments,  have  recently  been  published  in  an  issue  of  the  journal  Integrative 
Physiological  and  Behavioral  Science  (Vol  29,  No.  3, 1994). 

A  defining  feature  of  attractors  is  dissipation;  i.e.,  that  they  relax  to 
some  characteristic  state  following  perturbation.  There  are  formal  connections 
between  the  dissipative  and  attractor  generating  qualities  of  a  system  (Hale  & 
Koqak,  1991,  p.  394),  but,  as  discussed  a  little  later,  the  formalism  may  not  be 


applicable  to  many  biological  systems.  Given  these  constraints,  the  aim  will 
be  to  follow  the  notion  that  attractors  exist  based  on  the  observation  of 
relaxation  dynamics,  and  to  see  what  might  be  a  consequence  of  this 
dynamics,  rather  than,  for  example,  attempting  to  compute  its  fractal 
dimension.  A  number  of  studies  provide  examples  of  such  an  approach,  e.g., 
(Garfinkel,  1983;  Garfinkel,  et  al.,  1992;  Schiff,  1994;  Skarda  &  Freeman,  1987; 
Weiss,  et  al.,  1994). 

Pathways  for  communication.  Complex  systems  must  have  some 
method  for  distributing  information  among  their  components.  In  neural 
systems,  single  neurons  make  synaptic  connections  with  many  others; 
anatomically,  this  is  referred  to  as  divergence.  Conversely,  many  neurons 
make  synaptic  connections  onto  common  target  neurons;  this  is  referred  to  as 
convergence.  In  the  visual  cortex  of  mammals,  the  number  of  diverging 
connections  that  a  single  neuron  makes  is  in  the  order  of  5,000  to  10,000,  and 
each  neuron  receives  roughly  the  same  number  of  converging  inputs.  The 
situation  one  encoimters  in  studies  even  of  "simple"  invertebrate  animals,  such 
as  sea  slugs,  is  illustrated  in  Fig.  1  which  shows  that  a  few  cell  bodies  (shown 
by  the  bright  circular  objects  having  a  greenish  color)  send  tangles  of 
profusely  diverging  axons  and  neurites  (bright  green  strands)  throughout  a 
region  of  the  ganglion  known  as  the  neuropil  where  most  of  the  synapses 
between  neurons  take  place.  Some  of  the  neuron  cell  bodies  appear  greener 
than  others  because  they  contain  a  four-peptide  neurotransmitter  FMRF- 
amide.  The  axons  and  neurites  divide  into  thinner  and  thiimer  branches  as 
they  form,  what  seems  to  the  hiunan  eye,  a  entangled  web  in  the  neiuropil. 
Other  neurons  send  similarly  profuse  projections  to  converge  in  the  same 
areas.  How  such  extensive  connections  become  established  and  whether  there 
is  any  interrelationship  among  the  strengths  of  the  connections  are  questions 
that  remain  imanswered. 

HGURE 1  HERE 

Dynamic  readjustment  of  network  function.  An  additional  difficulty 
is  that  the  responses  of  biological  systems  are  often  short-lived,  variable, 
possibly  nonstationary,  and  are  continuously  subjected  to  many  different 
types  of  "external"  stimuli  that  perturb  them.  As  a  result,  the  functional  role  of 
a  neuron  in  an  assembly  may  be  vary  from  one  time  to  another,  and  may  best 
be  described  by  the  context  of  the  actions  expressed  by  other  individuals  with 
which  it  becomes  active  (for  discussions  and  a  review  of  some  of  the  literature 
on  the  subject,  see  Mpitsos  &  Soinila,  1993). 

Figure  2  provides  an  interesting  example  of  a  network  dynamically 
adjusting  to  a  perturbation.  The  illustration  shows  the  time  evolution  of 
electrical  potentials  obtained  from  neurons;  the  overall  pattern  relates  to 
feeding  behavior.  The  trace  labeled  BCN  is  from  a  neuron  that  takes  part 
(with  about  20  other  BCNs)  in  generating  the  pattern  of  activity  for  opening 

and  closing  of  the  jaws.l  The  term  "pattern"  refers  to  the  sequences  of  action 
potentials,  sharp  voltage  spikes  recorded  from  single  neurons  and  from  nerve 


roots.  This  cyclical  activity  is  generated  within  the  central  nervous  system;  i.e., 
it  is  a  central  pattern  generator  (CPG)  because  the  fundamental  oscillations  do 
not  depend  on  sensory  feedback.  When  the  BCN  is  forced  to  stop  firing  action 
potentials,  by  selectively  applying  hyperpolarizing  electrical  current  to  it 
(arrows  in  Fig.  2),  the  pattern  in  the  rest  of  the  traces  initially  stops  and  then 
restarts  while  the  BCN  is  still  suppressed.  Although  the  CPG  and  related 
activity  initially  depended  on  the  actions  of  the  BCN,  they  soon  recover  from 
the  loss.  How  such  reconfigurations  happen  is  a  an  important  and  interesting 
problem  facing  studies  of  the  complex  brains  in  mammals  (Lindsey,  et  al., 
1992),  as  well  as  of  "simple"  nervous  system  in  the  sea  slug. 

FIGURE  2  HERE 

The  themes  introduced  in  Figs.  1  and  2  are  observed  in  all  animals. 
Given  the  large  number  of  possible  pathways  that  activity  can  follow  from  one 
neuron  to  another,  the  activity  observed  at  different  times  may  be  variable  or 
unstable  (Adey,  1972;  Braitenberg,  1989;  Freeman,  1994;  John,  1972;  Lindsey  et 
al.,  1992;  Mpitsos,  1989;  Mpitsos  &  Cohan,  1986a;  Mpitsos  &  Cohan,  1986b; 
Mpitsos,  et  al.,  1988b;  Wu,  et  al.,  1989).  This  makes  it  quite  difficult  to  correlate 
the  activity  of  a  neuron  or  a  circuit  of  neurons  with  a  particular  function  or 
behavior,  suggesting  that  a  new  way  of  viewing  function,  some  new  language 
or  set  of  concepts,  is  needed  other  than  what  is  provided  by  the  idea  of  simple 
reflexes.  In  mammalian  studies,  Adey  (1972),  John  (1972),  and  Freeman  (see 
(Skarda  &  Freeman,  1987)  for  a  review)  were  perhaps  the  first  to  stress  the 
notion  of  functional  variability  in  neural  tissues.  Adey,  asks  whether  the  brain 
is  adaptively  a  "noisy  processor."  John  contrasts  the  notion  of  a  circuit  or 
"switchboard",  akin  to  the  circuits  in  a  computer  board,  against  statistical 
properties.  Freeman  (1994)  views  chaotic  variations  and  instabilities  as  a 
search  mechanism  by  which  neural  tissues  find  meaningful  firing  patterns  in 
changing  ensembles  of  neurons.  From  an  anatomical  perspective,  Braitenberg 
(1989)  speaks  of  the  neuron-to-neuron  firing  pathways  through  visual  cortex 
as  taking  a  random  walk.  Such  findings  indicate  that  we  must  re-examine  the 
notion  and  the  applicability  of  the  reflex,  defined  nearly  100  years  ago 
(Sherrington,  1906). 

Activity-dependent  changes  in  connection  strengths.  The  proposition, 
discussed  later,  is  that  attractors  emerging  from  a  population  of  neurons 
globally  cause  the  connections  within  this  population  to  change  in  such  a 
way  that  the  dynamics  is  strengthened.  To  implement  this  mechanism,  the 
connections  between  neurons  must  be  able  to  change  based  on  some  locally 
acting  rule  governed  by  the  firing  patterns  of  the  neurons.  It  has  been  shown 
in  biological  systems  that  at  least  some  synapses  are  established  by  activity- 
dependent  mechanisms,  e.g.,  (Kater  &  Mills,  1990;  Lipton  &  Kater,  1989; 
Mattson  &  Kater,  1989).  All  such  mechanisms  bear  some  similarity  to  a  rule 
stated  by  Hebb  (1949)  which  has  formd  considerable  utility  in  biological  and 
artificial  neural  network  studies.  It  states  that  if  there  is  appropriate  timing  in 
the  activity  between  two  connected  neurons,  the  synapses  undergo  metabolic 
or  structural  changes  that  affect  its  function. 


GRADIENTS  AND  FACTORS  THAT  CONTROL  THEM 

Phase  space  visualization  of  attractors.  Considering  systems  that  are 
definable  by  differential  equations,  there  are  three  types  of  attractore:  point 
attractors,  limit  cycles  (or,  more  generally,  periodic  and  quasiperiodic  cycles), 
and  chaotic  attractors.  A  rapid  way  to  examine  all  of  them  is  to  construct  phase 
portraits.  For  example,  to  examine  the  dynamics  of  the  movement  of  a  simple, 
friction-damped  pendulum  after  it  has  been  set  into  motion,  a  phase  portrait 
can  be  constructed  by  plotting  the  position  of  the  swing  of  the  pendulum 
against  its  velocity.  The  succession  of  the  position-velocity  states  from 
moment  to  moment  constitutes  a  trajectory  showing  how  the  system  evolves 
over  time,  though  time  is  an  implicit  variable  in  sud\  plots.  The  characteristic 
state  of  such  a  pendulum  is  the  resting  point  at  the  bottom  of  its  swing.  We 
may  think  of  the  momentary  stimulus  that  set  the  pendulum  into  motion  as  a 
perturbation  pulse.  The  element  that  dissipates  this  pulse  is  friction. 
Operationally,  the  dissipative  or  relaxation  process  appears  as  an  asymptotic 
spiral  or  gradient  leading  toward  the  resting  point  in  the  phase  plot.  Phase 
portraits  of  limit-cycle  attractors  contain  trajectories  that  form  a  single  closed 
loop;  other  periodic  attractors  have  closed  trajectories  with  two  or  more  loops. 
Phase  plots  of  chaotic  attractors  exhibit  infinitely  many  loops  of  the  trajectory, 
but  with  the  proviso,  as  required  by  the  Jordan  curve  theorem  and  the 
Poincare-Bendixson  theorem  (Hirsch  &  Smale,  1974;  Hofbauer  &  Sigmund, 
1988),  that  the  trajectory  never  closes  and  that  the  interrelated  positions  of  the 
loops  mix  in  a  certain  way,  e.g.,  (Andrade  et  al.,  1993;  Thompson  &  Stewart, 
1986,  p.  165). 

In  the  smooth  dynamics  of  systems  defined  by  differential  equation, 
one  speaks  of  flows  in  state  space.  Use  of  an  artificial  network  model  will 
serve  as  a  example  to  illustrate  phase  space  and  fixed  points.  The  exact 
specifics  of  this  system  have  been  described  in  detail  elsewhere  (Andrade  et 
al.,  1993).  Briefly,  it  is  composed  of  a  set  of  four  coupled  differential  equations 
that  describe  molecular  catalytic  action,  where  each  variable  represents  the 
concentration  of  one  of  the  catalytic  species;  a  single  species  may  catalyze  the 
replication  of  more  molecules  of  its  own  species,  or  of  other  species.  The 
mathematical  formulation  resembles  equations  describing  the  dynamic  of 
disparate  systems,  from  population  dynamics  to  neural  network  function 
(Cohen  &  Grossberg,  1983).  By  changing  the  parametric  setting  of  one  of  its 
catalytic  constants  (equivalent  to  changing  the  strength  of  a  neuronal  synapse, 
as  will  be  done  later  here),  the  system  generates  many  different  patterns  of 
activity  representing  all  three  t)^es  of  attractors  mentioned  above.  Such 
constants  are  called  bifurcation  parameters.  Changes  in  these  parameters 
usually  produces  little  change  in  system  dynamics,  but  at  certain  critical 
points  small  parameter  changes  cause  the  system  to  respond  qualitatively 
differently. 

The  operations  used  to  determine  the  presence  of  attractors  graphically 
are  illustrated  in  Fig.  3  for  the  catalytic  network  in  periodic  and  chaotic 


regimes  (only  species  XI  and  X2  are  shown).  The  time-evolution  of  the 
concentrations  in  both  regimes  are  shown  before  and  after  a  brief  perturbation 
was  applied  by  increasing  the  concentration  of  molecular  species  XI  for  a 
single  integration  step.  Since  XI  is  coupled  to  the  other  catalytic  species,  the 
pulse  of  externally  applied  extra  copies  of  XI  affects  the  concentrations  of  all 
species.  Following  the  perturbation  pulse,  the  periodic  oscillations  (but  not  the 
phase)  in  the  concentration  original  pattern.  The  method  has  been  widely 
used  in  studies  from  CPGs  in  neural  systems  to  circadian  rhythms  in  humans. 
A  similar  relaxation  process  occurs  in  the  chaotic  regime,  but  owing  to  the 
variations  that  occur  naturally  in  chaos,  the  recovery  is  difficult  too  assess 
using  time-series  data. 

FIGURE  3  HERE 

A  view  of  the  pre-  and  post-perturbation  conditions  is  more  clearly 
shown  graphically  by  the  phase  portraits  at  the  bottom  of  Fig.  3  than  by  ^e 
time-series  data  alone.  The  unperturbed  state  of  the  periodic  activity,  a  limit 
cycle,  is  shown  by  the  thick  trajectory.  The  thin-line  trajectories  show  the 
recovery  that  occurs  after  the  perturbation  stimulus  is  applied.  These 
trajectories  eventually  lead  back  to  the  original  limit  cycle.  In  the  chaotic 
regime,  the  relaxation  process  also  occurs,  but,  since  chaotic  processes  are 
sensitive  to  initial  conditions,  the  trajectory  never  returns  exactly  to  the 
original  thick  line;  i.e.,  the  attractor  is  represented  by  the  overall  structure  of 
the  phase  portrait  rather  than  a  particular  set  of  trajectories.  To  obtain  a  view 
of  the  gradient  fields  or  basins  of  attraction,  it  is  necessary  to  repeat  the  same 
experiment  many  times  using  different  concentrations  in  the  perturbation 
pulse;  and  pulses  may  be  applied  to  all  of  the  molecular  species,  as  well. 

In  the  pendulum  with  friction,  dissipation  occurs  through  heat  loss.  In 
the  catalytic  network,  dissipation  occurs  through  two  interrelated 
components.  In  one,  the  replication  process  is  error-prone,  and  error  copies  of 
all  of  the  different  molecular  species  are  ejected  from  the  network  into  an 
error-species.  The  second  component  arises  from  competition  between  the 
various  molecular  species  for  a  substrate  substance  from  which  they  make 
copies  of  themselves.  However,  the  total  number  of  molecules  of  all  species  in 
the  mixture  that  can  be  produced  by  replication  from  substrate  material  is 
capped  at  some  upper  limit.  The  pulse  of  externally  applied  molecules  of  XI 
causes  a  new  set  of  conditions  for  competition  for  substrate,  but  as  long  as  the 
excess  copies  are  present,  the  total  number  of  molecules  exceeds  the  capped 
limit.  The  relative  proportions  of  the  various  species  readjusts  nonlinearly 
since  the  extra  copies  of  XI  cause  the  synthesis  of  other  species,  so  adding  XI 
causes  a  redistribution  in  the  population  of  all  species.  At  the  same  time,  error 
copies  are  ejected  from  the  mixture,  slowly  redistributing  the  relative 
concentrations  progressively  back  to  the  distributions  relating  to  each 
dynamical  regimes. 

Fixed  points  control  system  responses  and  define  gradient  space. 
Fixed  points  represent  conditions  when  the  state  of  a  system  does  not 
changing  over  time  (Andrade  et  al.,  1993;  Seydel,  1988);  synonyms  are  steady 


states,  equilibria,  stationary  points.  The  quaUty  of  the  fixed  points  determines 
whether  they  attract  or  repel  trajectories  in  state  space.  Figure  4  illustrates  the 
relationship  between  the  trajectory  and  fixed  points  in  unperturbed  and 
perturbed  conditions  of  a  periodic  regime.  The  circles  show  four  of  the  points, 
all  saddles.  Point-1  is  the  only  fbced  point  at  which  all  four  species 
{X1,X2,X3,X4}  coexist.  For  some  parameter  settings,  the  point  is  stable, 
attracting  flows  to  it  in  all  directions  and  creating  conditions  in  which  the 
concentrations  of  all  species  remain  constant.  For  other  settings,  the  flows  are 
stable  toward  the  point  in  two  directions  and  unstable  in  the  other  two.  Points 
2  and  3  show  the  unstable  saddles  for  species-states  {X1,X2,X4}  and  {X1,X2), 
respectively.  Point-4  is  an  unstable  spiral  saddle  for  the  steady  state 
{X1,X2,X3},  and  as  in  points  2  and  3,  there  is  one  unstable  direction  of  flow. 
Stability  or  instability  along  each  axis  is  determined  by  the  eigenvalues  that 
characterize  the  quality  of  the  fixed  points.  Negative  eigenvalues  indicate 
stable  directions  of  flow;  positive  ones  indicate  unstable  ones. 

FIGURE  4  HERE 

The  quality  of  all  of  the  fixed  points  together  defines  the  characteristic 
features  of  the  potential  surface  of  attainable  concentrations.  This  surface 
consists  of  a  set  of  local  maxima  and  minima,  hills,  valleys,  saddles,  etc., 
comprising  a  landscape  whose  local  features  must  be  navigated  by  the 
dynamics  of  the  four  coupled  molecular  species.  The  effects  of  the  fixed  points 
extend  as  gradients,  so  the  course  of  nearby  trajectories  is  determined  by  the 
"push"  and  "pull"  of  all  the  fixed  points  that  exist  in  a  given  dynamical  regime. 
Consequently,  in  order  to  know  how  a  network  will  respond  imder  the 
variable  conditions  that  an  animal  may  experience  in  the  natural  environment, 
it  is  necessary  to  understand  what  the  gradient  fields  are  and  how 

perturbations  move  the  trajectory  around  them. 

Attractors  have  zero  volume.  The  attracting  limit  set  has  vamshingly 
small  volume.  This  is  indicated  by  the  spectrum  of  Lyapunov  exponents. 

These  provide  a  measure  of  the  rate  at  which  nearby  trajectories  converge  or 
diverge  with  respect  to  one  another.  There  is  an  exponent  for  each  dimension 
of  the  system.  For  the  system  to  remain  bounded,  the  sum  of  all  of  the 
Lyapunov  exponents  must  be  negative,  though  in  chaos  at  least  one  exponent 
must  be  positive  to  produce  stretching  along  a  ^ven  dimension,  e.g.,  see 
(Seydel,  1988;  Wolf,  et  al,  1985).  The  post-transient  behavior  of  the  attractor  is 
to  approach  zero-phase  space  volume.  The  rate  of  contraction  in  bits  per 

second  is  given  by  2(^1  +  ^2'^  ^3"^  ^4}^-  If  all  exponents  were  negative  (loss  of 
information  in  each  direction  of  flow),  the  activity  would  collapse  into  a  single 
point  in  all  directions.  In  the  example  shown  in  Fig.  4C,  the  exponents  are 
{+2.94, 0,  -49.06,-239.59}  (Andrade  et  al.,  1993).  The  positive  value  of  the 
principal  exponent  indicates  that  there  is  stretching  or  gain  of  information  in 
that  direction  of  phase  space.  The  next  exponent  is  vanishingly  small,  leading 
neither  to  expansion  or  contraction.  The  third  exponent  is  strongly  negative 
and  is  required  to  constrain  the  activity  in  phase  space.  The  fourth  exponent  is 


even  more  negative  than  the  third  such  that  information  in  this  dimension  is 
lost  extremely  rapidly.  The  exponents  of  periodic  activity  are  all  negative, 
except  for  the  first  which  is  zero.  So,  for  all  three  types  of  attractors,  there  is 
overall  contraction  in  phase  space. 

The  spectrum  of  Lyapimov  exponents  describes  the  local  flows  in  the 
system  and  is  an  invariant  measure  of  the  attractor,  regardless  of  the  wild 
swings  that  may  occur  in  the  overall  structure  of  the  imperturbed  attractor  in 
phase  space.  As  such,  they  provide  an  instantaneous  indicator  of  an  attractor, 
and  they  can  an  be  used  to  estimate  other  invariants,  such  as  the  attractor 
dimension  (Kaplan  &  Yorke,  1979;  and  see  examples  in  Andrade  et  al.,  1993; 
Wolf  et  al.,  1985).  Being  local-flow  measures  that  apply  to  the  attractor  as  a 
whole,  it  is  a  rather  interesting  speculation  that  they  may  provide  an 
immediate  way  for  a  network  to  detect  the  pattern  of  activity  it  receives  as 
opposed  to  detecting  it  based  on  the  large-scale  aspects  of  its  evolution  which 
would  require  much  longer  sampling  time. 

ATTRACTORS  IN  BIOLOGICAL  SYSTEMS:  SOME  CAVEATS 

Before  going  on  to  a  discussion  of  how  attractors  might  affect  networks, 
it  is  necessary  first  to  point  out  problems  or  constraints  that  must  be  consider 
when  positing  the  existence  of  attractors  in  biological  adaptation. 

Effects  produced  by  action-potential  activated  synapses.  Much  of 
what  is  known  formally  of  djmamical  systems,  and  partly  described  above, 
has  come  from  studies  of  differential  and  difference  equations,  or  of  systems 
that  can  be  put  in  these  forms.  In  neural  systems,  the  Hodgkin-Huxley  (1952) 
model  of  the  squid  axon  has  had  its  success,  and  fostered  much  research  on 
the  dynamics  of  nerve  membrane,  e.g.,  (Canavier,  et  al.,  1990;  Chay  &  Rinzel, 
1985;  Evans,  et  al.,  1982;  Fitzhugh,  1961;  Fitzhugh,  1969;  Jalife,  1990;  Rinzel  & 
Keller,  1973;  Rinzel  &  Miller,  1980)  partly  because  it  can  be  modeled  as  a 
system  of  differential  equations.  TTie  same  is  true  in  neural  network  studies 
when  the  communication  between  neurons  is  controlled  by  smoothly  graded 
synaptic  potentials  (Wang  &  Rinzel,  1992;  Rowat  &  Selverston,  1993),  and  in 
many  studies  dealing  wi^  psychophysical  problems  (Cohen  &  Grossberg, 
1983;  Grossberg,  1980;  Grossberg,  1988;  Zanone  &  Kelso,  1992). 

The  situation  becomes  much  more  complex  when  neurons 
communicate  with  one  another  using  the  pulses  of  action-potential  activated 
synapses.  A  major  reason  for  the  increase  in  complexity  is  that  the  differently 
timed  and  differently  shaped  S5maptic  inputs  act  as  perturbation  pulses  to 
subsequent  synaptic  inputs.  Even  for  a  simple  model,  such  as  the  catalytic 
network  discussed  above,  it  would  be  impossible  to  determine  how  the 
system  would  behave  in  response  to  a  series  of  differently  shaped 
perturbation  pulses  that  arrive  at  irregular  times. 

Instability  and  ergodicity.  The  goal-directed,  self-organizing  behavior 
of  biological  systems  can  be  imstable  (Freeman,  1994).  Examples  of 
spontaneous  shifts  of  motor  patterns  in  our  experimental  system  are  given  in 
Figs.  18  and  19  in  (Mpitsos  et  al.,  1988b).  Extended  discussions  of  the  problems 


may  be  found  in  (Mpitsos,  1989;  Mpitsos  &  Soinila,  1993).  Perhaps  the  most 
difficult  problem  to  address  is  the  finding  that  behaviors  and  motor  patterns 
may  blend  into  one  another  (Mpitsos  &  Cohan,  1986a;  Mpitsos  &  Cohan, 
1986b);  i.e.,  if  attractors  underlie  behaviors,  does  blending  imply  that 
attractors  blend  into  one  another  to  create  intermediates?  Even  if  the  number 
of  systems  that  exhibit  such  variabilities  is  small.  Freeman's  comments  are  a 
fair  warning  that  we  must  consider  instabilities  in  order  to  gain  a  fuller 
(alternative?)  understanding  of  self-organizing  behavior,  and,  especially,  how 
classical  dynamical  constructs  such  as  attractors  take  part  in  the  process,  or 
whether  they  apply. 

Ideal  multi-component  systems  in  which  transients  and  other 
instabilities  have  died  out,  and  in  which  all  available  states  are  equally 
probable,  may  be  thought  of  as  being  in  statistical  or  thermod5mamic 
equilibrium.  This  condition,  which  stems  from  ergodic  theory  (Eckmann  & 
Ruelle,  1985;  Friedman,  1970;  Smale,  1980),  may  provide  a  principle  that  can 
be  used  to  understand  how  complex  biological  systems  organize  globally.  It  is 
necessary  to  consider  ergodicity  because  the  aim  here  will  be  to  say  that  the 
emergence  of  the  "cooperative"  assembly  of  neurons  is  a  post-perturbation 
residue  that  emerges  from  a  larger  population  through  a  common  driving 
force.  This  force  is  the  gradient  action  of  the  attractor. 

Ergodic  conditions  are  difficult  to  prove  in  model  systems,  let  alone  in 
biological  ones.  A  simple  model  system  can  be  nm  long  enough  that  it  closely 
approximates  statistical  equilibrium,  but  biological  systems,  almost  by 
definition,  are  far  from  equilibrium.  During  embryonic  development,  and 
learning,  and  in  the  normal  moment  to  moment  states  that  do  not  involve 
development  or  learning,  biological  systems  are  continuously  bombarded  by 
many  extraneous  stimuli;  biological  systems  are  always  changing  and  are 
subjected  to  many  perturbations  such  that  asymptotically  stable  conditions 
seldom  arise.  It  is  necessary,  therefore,  to  soften  the  statement  of  ergodicity  to 
say  that  gradient-seeking  activity  in  biological  systems  tends  toward  ergodic 
conditions,  though  it  may  be  far  off.  Similarly,  it  is  necessary  to  speak  not  of 
an  attractor,  which  is  an  ideal  equilibrium  condition  reached  asymptotically, 
but  rather  of  gradient  behavior  as  the  state  of  a  system  tends  or  is  drawn 
toward  the  attractor. 

Does  simple  function  emerge  from  complex  structure?  Although  there 
are  only  about  20  BCNs  on  each  side  of  the  bilaterally  s)nnmetric  nervous 
system  of  Pleurobranchaea,  they  are  embedded  in  larger  networks.  In  the 
motor  ganglion  in  which  the  BCNs  are  located,  there  are  roughly  400  neurons, 
some  of  which  are  shown  in  Fig.  1.  The  BCNs  connect  to  other  neurons  in  this 
ganglion  and  in  other  ganglia,  and  receive  feedback  from  them  directly  or 
indirectly  through  other  neurons.  Overall,  several  thousand  neurons,  located 
in  different  parts  of  the  nervous  system,  may  take  part  in  generating  the 
pattern  of  neural  activity  underlying  a  bout  of  feeding  behavior.  However, 
many  oscillatory  patterns  (thou^  not  all)  that  are  generated  by  such  networks 
recover  rapidly  from  perturbations.  Although  phase  information  may  be  lost. 


biological  systems  are  observed  to  be  homeostatic,  and  even  if  many  different 
states  are  possible,  small  perturbations  of  a  given  state  usually  leads  to  its 
recovery.  Because  of  such  a  relaxation  process  there  is  a  tendency  for  the 
activity  representing  the  attractor  to  contract  spatially.  This  suggests,  as  in  the 
case  of  the  BCNs  in  Fig.  2,  that  fewer  than  the  several  thousand  neurons  that 
are  active  during  the  feeding  motor  pattern  may  be  required  to  generate  the 
dynamics  underling  the  behavior.  This  relates  to  the  above  idea  that  the  sum 
of  the  Lyapunov  exponents  must  be  negative  for  the  system  to  remain 
bounded;  overall,  there  must  be  contraction  in  phase  space  in  the  responses  of 
individual  neurons,  and  in  the  interrelated  firing  between  neurons.  The  notion 
of  spatial  contraction  is  used  in  the  sense  of  exclusion  from  the  final 
cooperative  ensemble.  It  does  not  imply  that  the  final  generative  set 
necessarily  consists  of  adjacently  located  cell  bodies.  The  excluded  neurons 
may  become  quiescent  or  they  may  continue  to  fire  independently  of  the 
attractor,  or  they  may.  be  driven  by  the  attractor  yet  may  not  be  part  of  the 
neural  machinery  that  generates  the  attractor. 

Having  said  all  that,  it  is  necessary  to  point  out  that  fundamental 
attractor  features,  such  as  fixed  points,  are  not  definable  in  most  biological 
networks  that  rely  on  action  potentials  for  communication  between  neurons, 
even  in  simple  model  networks  such  as  the  ones  discussed  below.  Neither  do 
we  understand  the  specifics  of  how  different  spatio-temporal  regimes  emerge 
or  how  to  define  them  from  the  perspective  of  how  individual  neurons  take 
part  in  the  process,  though  considerable  work  has  been  done  along  that  line, 
e.g.  (Freeman,  1994;  Skarda  &  Freeman,  1987).  Neither  is  it  fully  understood  as 
to  how  to  assess  the  state  of  a  system  or  attractor  quantitatively.  The 
publication  by  Grassberger  and  Procaccia  (1983)  of  a  method  for  estimating 
attractor  and  embedding  dimensions  provided  one  easily  applicable  avenue 
for  making  assessments  at  least  for  the  number  of  variables  or  dimensions  that 
may  sustain  an  attractor.  Over  the  years,  a  number  of  important  refinements 
have  been  proposed  to  improve  the  methods  for  determining  attractor 
dimensions,  whether  chaos  is  present,  or  wheflier  there  are  nonlinearities  in 
the  data,  e.g.,  (Grassberger,  1990;  Judd,  1992;  Kaplan,  1993;  Kaplan  &  Glass, 
1992;  Mayer-Kress,  1986;  Schiff,  et  al.,  1994;  Skinner,  et  al.,  1994;  Theiler,  1986; 
Theiler,  1987;  Theiler,  1993;  Theiler,  et  al.,  1992),  but  as  Theiler  (1994)  has 
pointed  out,  much  "black  art"  is  involved.  Moreover,  as  Theiler  and  Rapp 
(1995),  Mayer-Kress,  Barczys  and  Freeman  (1991),  Glass  and  Kaplan  (1993) 
point  out,  evidence  for  low  dimensionality  and  chaos  in  measurements  of 
biological  activity  is  inconclusive.  Although  electrical  stimulation  methods, 
potentially  applicable  to  the  control  of  epileptic  foci,  have  been  developed  that 
use  the  characteristics  of  unstable  fixed  points  to  obtain  evidence  both  of 
periodic  activity  and  chaos  in  hippocampal  slices  (Schiff,  1994),  the  majority  of 
the  quantitative  assessments  of  activity  in  the  same  tissue  point  to  stochastic 
processes  rather  than  to  low-dimensional  chaos  (Schiff  et  al.,  1994).  In  our  own 
studies,  the  evidence  for  chaos  was  state  explicitly  as  being  only 
circumstantial  (Mpitsos,  et  al.,  1988a;  Mpitsos  et  al.,  1988b).  Although  we  have 


not  yet  exhausted  the  available  methods  on  estimates  of  low  embedding 
space,  e.g.,  such  as  the  neural  network  methods  described  in  (Abarbanel  et  al., 
1993),  use  of  surrogates  (Theiler,  1994)  indicates  that  though  there  is  structure 
in  the  firing  patterns  of  individual  BCNs  and  motor  neurons  that  they  drive, 
the  evidence  for  low-dimensional  embedding  space  is  also  wanting.  This  is 
remarkable  because  the  neurons  we  have  examined  and  the  neural  firing 
patterns  that  they  generate  are  related  to  identifiable  behaviors,  whereas  the 
firing,  for  example,  of  compoimd  action  potentials  in  hippocampal  slices 
(Schiff  et  al.,  1994)  need  not  and  probably  do  not  represent  any  particular 
behavior.  Moreover,  in  our  experience,  motor  patterns  appear  less  variable 
when  obtained  from  isolated  nervous  systems  than  in  whole  animals, 
indicating  that  the  numerical  evidence  for  low  dimensional  (periodic  or 
chaotic)  attractors  in  whole  animals  should  be  even  less  conclusive. 

The  problem  in  our  own  studies,  and  in  other  studies  mentioned  above, 
may  be  only  one  of  quantitative  verification,  for  which  the  appropriate 
methods  have  not  been  developed  yet,  or  that  we  may  not  have  conducted 
experiments  that  yielded  the  appropriate  data.  Alternatively,  perhaps  the 
search  for  crisply  definable  determinism  may  be  equivalent  to  the  search  to 
identify  a  network  anatomically  as  the  neural  correlate  of  behavior  (Mpitsos  & 
Soinila,  1993).  But,  following  Adey  (1972)  and  John  (1972),  it  may  be 
worthwhile  to  consider  further  the  utility  of  stochastic  processes.  One  of  the 
reasons  may  be  that  while  synaptic  transmission  involving  action-potential 
evoked  synapses,  as  opposed  to  smooth  graded  transmission,  may  be 
deterministic,  it  also  appears  that  degeneracy  of  information  flow  across  the 
synapse  may  also  occur  (Edstrom  &  Mpitsos,  1995b). 

The  dynamics  of  spatio-temporal  recovery  may  also  not  be  simple.  We 
can  not  just  say  that  the  recovery  process  reestablishes  the  set  of  neurons 
generating  the  attractor  before  the  system  was  perturbed  and  leave  it  at  that. 
The  reason  for  this  is  that  the  generation  of  an  attractor  may  not  require  the 
activity  in  any  particular  neuron.  Even  fewer  than  the  20  BCNs  may  be 
necessary  for  generating  the  CPG  or  the  behaviors  that  the  CPG  drives.  The 
effect  shown  in  Fig.  2  is  an  extreme  example  of  this.  Other  examples,  not 
presented  here,  show  that  the  interrelated  firing  of  neurons  in  a  cooperative 
set  may  vary  over  time,  and  the  removal  of  some  BCNs  may  have  essentially 
no  effect  on  the  oscillatory  patterns  (Mpitsos  &  Cohan,  1986b).  In  a  sense, 
there  is  a  kind  of  redundancy  in  a  population  of  neurons  that  generate 
attractors.  Up  to  a  point  one  can  remove  neurons,  and  the  system  can 
rearrange  itself  to  give  the  same  attractor  as  before,  though  eventually  a 
minimal  population  may  be  reached  from  which  we  can  not  remove  neurons 
and  retain  the  attractor  (Selverston,  1980;  Selverston,  1993;  Rowat  & 

Selverston,  1993).  Under  the  most  general  conditions,  the  terminal  group  may 
be  different,  depending  on  which  neurons  were  removed  at  the  beginning. 
Therefore,  in  the  recovery  of  attractor  systems,  the  crucial  element  is  not 
necessarily  what  neurons  remain,  but  whether  certain  gradient  can  be 
sustained. 


ATTRACTORS  IN  SIMULATION  OF  BIOLOGICAL  NETWORKS 

The  network  shown  in  Fig.  5C  will  be  examined  here  and  used 
subsequently  to  examine  how  activity  affects  network  structure.  This 
particular  architecture  was  chosen  as  part  of  a  larger  experiment  aimed  at 
studying  both  periodic  and  complex  firing  patterns,  though  only  simple 
patterns  will  be  examined  here.  The  network  consists  of  three  cells;  Cell4  is 
external  to  the  network  because  it  receives  no  feedback  from  the  other  cells.  A 
number  of  network  constants  that  may  be  used  as  control  parameters  with 
which  to  produce  "bifurcations",  qualitative  changes  in  the  interrelated 
patterns  of  action  potentials  produced  by  the  three  cells:  (a)  Changing  the 
firing  level  of  Cell4  by  injecting  it  with  different  sustained  levels  of 
depolarizing  electrical  current,  (b)  Keeping  Cell4  quiescent,  but  injecting 
depolarizing  current  into  one  of  the  other  cells  in  the  network  to  adjust  its 
firing  level,  (c)  Or  by  selectively  changing  the  synaptic  strength  of  one  of  the 
synapses. 

FIGURE  5  HERE 

Figure  5 A  shows  a  3-cycle  firing  pattern  in  Cell2.  Starting  from  the 
simulation  time  at  680  msec,  seven  intervals  between  the  action  potentials  are 
shown,  where  the  3-cycle  repetition  sequence  consists  of  intervals  having  15.4 
msec,  35.0  msec,  and  46.3  msec  durations.  For  the  purposes  of  the  illustration 
only  two  3-cycle  periods  are  shown,  but  many  more  3-cycle  sequences 
preceded  them.  This  particular  firing  pattern  was  obtained  by  driving  the 
network  using  steady  depolarization  of  Celll  and  selectively  adjusting  the 
strength  of  the  inhibitory  synapse  that  Cell3  makes  on  Cell2. 

The  values  for  the  synaptic  strengths  giving  this  3-cycle  are  illustrated 
in  Fig.  5C,  but  certain  other  combinations  of  synaptic  weights  give  similar  3- 
cycle  activity,  just  as  Gardner  (1990)  has  demonstrated  that  different  synaptic 
strengths  between  four  identified  neurons  in  the  sea  slug  Aplysia  generate 
similar  network  function.  The  dashed  line  below  the  action  potential  trace 
shows  the  time  during  which  the  network  was  perturbed  by  injecting  Cell2 
with  a  125  msec  electrical  current  pulse  that  pushed  the  membrane  potential 
below  the  threshold  level  for  the  genesis  of  action  potentials,  except  for  one 
action  potential  (labeled  by  a  "1")  arising  from  strong  activity  in  one  of  the 
other  cells  in  the  network.  After  the  cessation  of  the  applied  current,  the  cell 
rebounded  with  a  rapid  burst  of  three  action  potentials  (labeled  "2",  "3",  "4"), 
and  then  recovered  to  its  original  firing  pattern. 

The  pattern  of  intervals  shown  in  Fig.  5A  may  be  replotted  in  a  return 
map  in  which  one  interval  in  the  sequence  is  plotted  on  the  horizontal  axis 
and  the  succeeding  interval  is  plotted  on  the  vertical  axis.  One  hundred 
intervals  were  used  to  construct  the  3-cycle  of  the  vmperturbed  activity;  all  fell 
sequentially  on  the  three  filled  circles  in  Fig.  5B.  The  return-map  positions  of 
the  intervals  during  and  after  the  application  of  the  125  msec  perturbation 
pulse  are  shown  by  the  numbered  and  open  triangles.  The  identifying 
numbers  in  the  return  map  are  the  same  as  those  in  the  trace  shown  in  Fig.  5A, 


where  the  number  over  a  given  action  potential  indicates  the  interval  between 
that  action  potential  and  the  preceding  one. 

What  causes  dissipation  in  biological  networks?  The  friction-damped 
pendulum  and  the  catalytic  network  described  above  have  known  dissipative 
mechanisms  with  which  to  conceptualize  the  recovery  process,  but  there  is  no 
obvious  anchor  in  networks  such  as  the  one  shown  in  Fig.  5C.  It  is  not  simply 
that  there  are  inhibitory  synapses  in  the  network.  Although,  the  excitatory  and 
inhibitory  synapses  were  chosen  to  produce  expansion  and  contraction 
qualities  in  the  firing  patterns  among  the  three  neurons,  modeled  after  Rossler 
(1976),  dissipative  firing  can  be  obtained  by  using,  for  example,  only 
excitatory  synaptic  cormections.  An  understanding  of  the  cellular  processes 
underlying  dissipative  action  in  biological  systems  may  shed  light  in  how 
individual  neurons  take  part  in  group  action  (Edstrom  &  Mpitsos,  1995a),  but 
the  subject  has  been  largely  bypassed  in  neurobiological  applications  of 
dynamical  theory. 

Effect  of  gradient-seeking  activity  on  biological  networks.  Gradient- 
seeking  behavior  in  neural  networks  has  been  a  subject  of  interest  for  a  long 
time,  and  a  large  literature  has  emerged,  e.g.,  (Cohen  &  Grossberg,  1983; 
Hopfield,  1982;  Le  Cun  et  al.,  1991;  Perlmutter,  1992;  Plaut,  et  al.,  1986; 
Rumelhart,  et  al.,  1986;  Werbos,  1974;  Widrow  &  Steams,  1985).  The 
possibilities  that  gradients  may  change  biological  networks  is  considered  in 
the  following  scenario: 

(1)  Assume  that  the  network  is  initially  randomly  connected,  and  that 
the  neurons  are  firing  action  potentials. 

(2)  Somewhere  in  the  network,  two  or  three  neurons  may  have  firing 
patterns  that  are  sufficiently  correlated  with  one  another  that  their  synapses 
undergo  activity-dependent  changes.  If  dissipative  action  begins  to  emerge, 
the  synaptic  weights  in  the  developing  network  must  have  some  interrelated 
range  of  values,  else  the  attractor  would  not  exist. 

(3)  The  emerging  attractor  constrains  the  flow  of  activity,  causing 
further  activity-dependent  changes  and  refinement  of  the  attractor 
spatiotemporally. 

(4)  Neurons  that  converge  onto  the  same  follower  neurons  may 
compete  (Merzenich,  et  al.,  1983a;  Merzenich,  et  al.,  1983b;  Merzenich,  et  al., 
1984;  Mpitsos  et  al.,  1978)  for  occupancy  of  the  synaptic  space  available  on  the 
follower  neuron  such  that  connections  may  be  made  or  broken,  structurally 
molding  the  neuroarchitecture;  i.e.,  attractors  are  the  ultimate  architects  of 
neural  tissues  (or  of  any  biological  system  governed  by  gradient-seeking 
behavior). 

A  concrete  example  of  this  would  be  useful  to  visualize  the  process. 
Our  work  on  this  is  very  preliminary,  but  it  is  worth  discussing  to  illustrate 
the  above  conjecture.  The  schematic  in  Fig.  6  shows  how  the  network 
examined  previously  may  be  altered  following  a  perturbation  stimulus.  The 
original  network  is  shown  in  Fig.  6A.  It  is  the  same  as  the  one  shown  in  Fig.  5, 
as  are  the  activation  conditions,  but  the  strengths  of  the  synapses  are  now 


made  available  for  activity-dependent  changes.  If  Cell2  is  depolarized  for  an 
arbitrary  length  of  time,  it  begins  to  generate  action  potentials  initially  at  a 
higher  frequency  than  before,  but  soon  the  balance  of  excitatory  and  inhibitory 
feedback  readjusts  its  rate  of  firing.  The  activity-dependent  changes  in  a 
synapse  depend  on  the  temporal  correlations  of  activity  in  the  presynaptic  and 
postsynaptic  cells  which  will  vary  over  time.  The  dynamics  giving  rise  to  these 
correlations  is  extremely  complicated  and  time  dependent,  but  Fig.  6B  has 
caught  the  changed  synapses  at  a  stage  (shown  by  the  numbers  next  to  the 
axons)  that  cause  Cell2  to  generate  a  6-cycle  firing  pattern.  The  open  circles  in 
the  return  map  of  Fig.  6C  show  the  values  of  the  six  intervals  between  the 
action  potentials.  The  filled  circles  show  the  original  3-cycle. 

FIGURE  6  HERE 

When  Cell2  is  released  from  the  depolarizing  current  (Celll  is  under 
continuous  depolarization  to  keep  the  network  active),  its  firing  pattern  may 
remain  in  the  6-cycle  state,  but  if  the  attractor  is  not  sufficiently  well  formed, 
genesis  of  action  potentials  in  the  three  neurons  may  drift  to  some  other  set  of 
patterns,  or  it  may  eventually  settle  back  to  the  original  3-cycle  state  (and 
perhaps  with  a  different  set  of  synaptic  strengths  between  the  neurons). 

The  aim  here  is  not  to  say  that  the  6-cycle  per  se  is  formed,  nor  that  the 
original  3-cycle  is  regenerated  in  the  recovery,  but  that  some  attractor  is 
generated,  and  that  its  gradients  are  what  adjusts  or  globally  "directs"  the  local 
activity-dependent  synaptic  changes.  This  is  different  than  would  be  imagined 
were  we  to  view  networks  as  switchboards  or  reflexes  by  which  change  is 
based  only  on  local  activity-dependent  effects.  From  such  effects  it  might  be 
postulated,  for  example,  that  associative  learning  in  whole  animals  is  based  on 
changes  in  individual  synapses;  reviewed  in  (Mpitsos  &  Lukowiak,  1985).  By 
the  conjectured  gradient  method,  however,  it  is  obvious  that  changing  one 
synapse  causes  changes  in  all  of  them.  Local  rules  are  a  work,  but  they  are 
guided  by  a  more  global  principle. 

GRADIENTS  OF  LEARNING  IN  ARTIFICIAL  NEURAL  NETWORKS 

The  above  ideas  are  derived  from  the  extensive  studies  on  artificial 
networks,  though  their  effect  is  only  beginning  to  become  apparent  in 
biological  studies.  I  shall  discuss  the  results  of  simple  artificial  neural  network 
studies  (Burton  &  Mpitsos,  1992;  Mpitsos  &  Burton,  1992),  based  on  the 
backpropagation  algorithm  (Rumelhart  et  al,  1986),  to  examine  possibilities 
that  might  not  have  been  conceived  from  studies  on  biological  networks.  The 
backpropagation  algorithm  relies  on  a  negative-gradient  mechanism  to 
minimize  error  as  synapses  and  thresholds  are  changed  during  training. 
Gradner  (1993)  has  suggested  cellular  mechanisms  by  which  backpropagation 
might  take  place  in  biological  systems,  and  backpropagation  methods  have 
been  used  to  address  specific  cases  in  biology,  such  as  the  mechanism  of  local 
bending  in  the  leech  (Lockery  &  Sejnowski,  1992).  However,  rather  than 
addressing  a  particular  application  or  biological  system,  I  shall  examine  some 


of  the  more  rudimentary  effects  that  might  arise  generally  in  networks  from 
gradient-seeking  mechanisms. 

The  network  examined  here  is  shown  in  Fig.  7(Top).  The  input  consists 
of  a  single  neuron  whose  synapses  are  distributed  to  a  set  of  "intemeurons",  or 
hidden  neurons,  labeled  h.  The  number  of  hidden  units  varies,  depending  on 
the  experiments.  The  hidden  units  converge  onto  a  single  output  neuron.  The 
input  signal  consisted  of  a  sequence  of  discrete  values  generated  by  the 
chaotic  logistic  function/(x)  =  3.95(l-x)x,  where  x  is  in  the  unit  interval.  One  of 
the  aims  of  the  original  studies  was  to  determine  whether  a  single-input 
network  with  no  bypass  coimections  could  predict  a  chaotic  signal,  but  in 
order  to  keep  the  illustrations  as  simple  as  possible,  the  discussion  here  is  of 
training  using  the  identify  function  as  the  teacher  whereby  the  output  is 
required  only  to  reproduce  the  input.  Despite  its  simplicity,  the  identity 
function  provides  insight  into  the  effects  of  gradients  and  permits  the  use  of 
simple  graphics  to  demonstrate  effects.  An  on-line  training  method  was  used 
in  which  synapses,  and  threshold  levels,  are  adjusted  after  each  training  trial. 
In  this  way  the  network  is  given  information  only  of  a  single  error-point 
relating  to  the  gradient  fields  of  the  teacher  fimction.  Batch  training  updates 
the  synapses  after  a  number  of  trials,  providing  more  information  about  the 
teacher  function  than  on-line  training. 

FIGURE  7  HERE 

Gradient  descent  during  learning  adjusts  synaptic  strengths 
optimally  with  respect  to  one  another.  The  network  was  trained  until  the 
difference  between  its  output  and  the  teacher  requirement  converged  to  some 
minimum  error.  The  synapses  and  thresholds  were  frozen  at  the  trained 
values,  and  training  effects  were  excluded  during  subsequent  mampulations 
of  the  network.  One  synapse  at  a  time  was  then  set  at  a  series  of  arbitrarily 
selected  values  while  holding  all  other  synapse  at  their  trained  value.  For  each 
value,  the  network  was  presented  a  number  of  input  values  of  the  logistic  to 
obtain  the  average  error  as  a  function  of  the  changes  in  each  synapse  (Fig.  7 
Bottom).  As  should  be  expected  of  backpropagation  training,  the  error- 
gradients  generated  during  training  set  the  synapses  optimally  with  respect  to 
one  another,  as  was  proposed  for  biological  simulations  in  Fig.  6. 

Two-dimensional  error  gradients  or  surfaces  may  be  constructed  using 
by  measuring  the  error  as  a  function  of  changes  in  two  synaptic  weights.  The 
asterisk  in  Fig.  8  shows  the  location  of  the  two  weights  that  gave  the  minimum 
error.  By  adding  a  third  synapse,  the  gradients  become  more  complex,  as 
shown  in  Fig.  9,  where  color  represents  error.  Minimum  error  is  somewhere 
inside  the  box  of  Fig.  9A.  A  series  of  slices  through  the  box  (Fig.  9B)  reveals  the 
location  of  the  minimum  error  which  is  indicated  by  deep  blue  in  the  top 
horizontal  slice.  This  slice  is  identical  to  the  surface  shown  in  Fig.  8;  the  dark 
blue  spot  in  the  middle  corresponds  to  the  asterisk  in  Fig.  8.  Different 
initialization  parcimeters  yield  a  different  set  of  final  optimal  weights,  as 
occurred  in  the  simulations  in  Figs.  5  and  6,  and  as  reported  in  comparisons  of 


the  connections  between  a  set  of  four  identifiable  neurons  in  different 
specimens  of  the  sea  slug  Aplysia  (Gardner,  1990). 

FIGURES  7, 8  &  9  HERE 

Gradients  limit  the  number  of  neurons  in  a  cooperative  assembly. 

The  common  intuitive  impression  of  the  effect  of  the  size  of  the  network  is 
"the  bigger,  the  better."  This  proved  to  be  true  to  some  extent  (Mpitsos  & 
Burton,  1992),  but  after  some  optimal  small  number  of  hidden  units  was 
reached,  increasing  the  number  further  rapidly  led  to  learning  failure.  The 
more  complicated  the  learning  task,  the  sooner  the  detrimental  effects  occur, 
even  for  learning  the  identity  function. 

There  are  at  least  three  interrelated  factors  underlying  size  limitation, 
all  related  to  gradients.  The  top  horizontal  slice  in  Fig.  9B  contains  obvious 
gradients  leading  to  an  optimal  solution  of  for  weights  {W1,W2,W3};  other 
slices  contain  large  areas  of  flat  gradients.  Comparing  Figs.  7-9  shows  that 
changing  from  1  to  3  degrees  of  freedom  increases  the  possibility  of 
encoimtering  large  flat  gradient  fields.  As  the  number  of  trainable  synapses  or 
units  increases,  the  surroimding  high-error  region  grows  exponentially  by 
comparison  to  the  central  low  error  (Fig.  10).  This  makes  the  distances  that  the 
network  must  traverse  through  weight-error  space  from  high  too  low  error 
during  training  extremely  long  and  the  gradients  flat  in  all  directions. 

FIGURE  10  HERE 

Contributing  to  size  limitation  is  the  on-line  training  method  since  in 
each  training  trial  the  network  is  exposed  to  a  single  error-point  rather  than  to 
the  structure  of  the  entire  error  surface.  The  error  fields  in  Fig.  7-9  were 
constructed  artificially  after  the  network  was  trained,  but  the  training  method 
contained  no  information  about  the  structure  of  the  error  fields,  making 
learning  difficult. 

The  third  factor  limiting  network  size  is  introduced  by  a  type  of 
"stiffness"  in  gradient  decent.  This  arises  from  cross  correlations  in  the  input 
matrix  to  neurons  that  receive  multiple  synapses  (for  a  detailed  discussion  see 
Le  Cun  et  al.,  1991).  If  no  cross  correlations  exist,  the  rate  of  gradient  decent  is 
the  same  in  all  directions,  generating  symmetric  error  fields  in  all  directions. 
Ideally,  minimum  error  can  be  reached  in  a  single  time  step.  If  correlations 
exist,  the  rate  of  descent  is  not  symmetric  causing  distortion  and  stretching  of 
error  fields.  Analogous  stiffness  occurs  in  numerical  integration  of  differential 
equations  having  disparate  time  scales.  High-dimensionality  and  complex 
teachers  (gradients)  exacerbate  this  problem,  slowing  or  preventing  learning. 

Biological  systems  are  inherently  high-dimensional,  far  from  ideal,  and 
probably  possess  conditions  for  analogs  of  stiffness  to  exist.  The  neural 
network  findings  suggest  that  the  rate  at  which  coherent  activity  emerges  in 
large  biological  networks  will  be  slower  than  in  smaller  networks.  In 
embryonic  development,  for  example,  certain  rules  (perhaps  genetic  ones) 
may  limit  the  number  of  neurons  in  a  cooperative  assembly,  but  these  rules 
may  be  assisted  by  the  limitations  imposed  by  gradient  descent  into  attractors. 
Smaller  populations  of  neurons  may  reach  "consensus"  to  generate  an  attractor 


faster  than  much  larger  ones,  and  may  generate  stronger  attractors  since 
smaller  networks  may  have  less  chance  of  falling  into  non-optimal  connection 
configurations  that  generate  weak  attractors.  It  is  perhaps  not  just  a  humorous 
exaggeration  to  extend  the  notion  of  stiffness  to  consensus-building  or  lack 
thereof  in  university  faculty  meetings  and  in  the  United  States  Congress. 

Adjustable  thresholds  are  essential  for  learning  to  occur .  The 
presence  of  adjustable  response  thresholds  in  neurons,  in  addition  to  the 
presence  of  adjustable  synaptic  strengths  between  neurons,  is  necessary  for 
leanung  to  occur  properly  (Mpitsos  &  Burton,  1992).  The  above  mentioned 
deficits  relating  to  stiffness  can  be  largely  overcome  for  simple  learning  tasks 
in  our  network  by  slowing  the  rate  of  learning  in  the  output  umt,  but  even  this 
does  not  make  up  for  the  reported  deficits  that  emerge  when  networks  do  not 
contain  trainable  thresholds.  Recent  studies  (Leshno,  et  al.,  1993)  provide  a 
general  proof  for  the  necessity  of  thresholds  in  certain  artificial  networks,  and 
findings  in  networks  of  biological  neurons  in  culture  (Turrigiano,  et  al.,  1994) 
indicate  that  adjustments  in  firing  thresholds  may  stabilize  firing  patterns. 

The  facilitating  effect  of  thresholds  arises  from  that  fact  that  they  act  as 
a  bias  (Rumelhart  et  al.,  1986).  This  bias  shifts  the  mid-region  of  the  sigmoidal 
output  functions  selectively  to  different  regions  on  the  activation  axis  (Fig.  11; 
thin-line  curves),  allowing  the  network  to  reach  activation  states  that  would 
orUy  be  asymptotically  reachable  in  the  absence  of  thresholds  (Fig.  11;  thick¬ 
line  curve!  The  activation  of  an  output  unit  may  be  thought  of  as  a 
polynomial  function  of  the  inputs  from  the  hidden  units.  Selective  control  of 
all  thresholds  allows  fine  adjustments  to  the  polynomial,  allowing  network 
function  to  conform  to  the  error-field  requirements  of  the  learning  task.  Given 
that  attractors  in  biological  systems  may  have  considerably  more  complicated 
gradients  than  the  identity  function  shown  here,  it  is  expected  that  adjustable 
thresholds  may  be  important  when  learning  involves  attractors.  However, 
compared  to  the  extensiveness  of  studies  on  synaptic  changes  during  learning 
in  biological  systems,  little  attention  has  been  paid  to  threshold  changes  and 
how  such  changes  affect  the  computational  ability  of  networks. 

HGURE  11  HERE 

Networks  encode  multiple  forms  of  information.  The  term  encoded 
information  typically  refers  to  processes  that  are  evoked  among  neurons  in  a 
network  to  produce  an  output.  The  output  of  the  network  can  be  sent  to  other 
networks  for  further  computation  or  for  producing  behavior.  A  given  network 
may  contain  the  information  for  generating  one  or  more  processes  (Mpitsos  & 
Cohan,  1986a;  Mpitsos  &  Cohan,  1986b;  Sejnowski  &  Rosenberg,  1987;  Skarda 
&  Freeman,  1987).  Ultimately,  this  information  is  "laid  down  in  structure" 
(Lorenz,  1974),  such  as  in  the  strength  of  synapses,  in  the  structure  of  the 
neurocircuit  itself,  and  even  in  changes  relating  to  gene  expression. 

Networks  may  contain  a  second  form  of  encoded  information  that  does 
not  become  obvious  until  networks  are  required  to  learn  something  new 
(Burton  &  Mpitsos,  1992).  In  these  experiments,  noise  was  injected  into  the 
adjustments  that  were  applied  to  the  synapses  and  thresholds  in  order  to 


improve  the  rate  of  learning.  Interestingly,  it  also  optimized  the  rate  of 
learning  of  a  subsequent  task  when  noise  was  not  given.  Networks  that  were 
not  given  noise  during  the  first  training  session  eventually  learned  the  first 
task  equally  as  well  as  ones  that  received  noise  (i.e.  all  networks  reached  the 
same  level  of  performance),  but  they  learned  the  second  task  more  slowly. 

To  account  for  this,  consider  the  error  surfaces  in  Figs.  8  and  9.  Without 
noise,  the  trajectory  through  weight-error  space  flows  relatively  smoothly, 
covering  only  a  small  region  the  potential  errors.  With  noise-adjusted 
synapses,  the  network  is  forced  to  produce  large  error  shifts  during  which  the 
trajectory  covers  a  relatively  large  portion  of  the  potential  error  space.  The  two 
networks  reach  the  same  minimum  error,  but  sample  the  error-space 
differently.  This  suggests  that  the  second  form  of  information  contained  in  a 
set  of  connections  may  relate  to  not  so  much  to  the  actual  final  task  but  to  the 
error  gradients  that  must  be  navigated  during  learning. 

If  new  tasks  or  teacher  fimctions  involve  error  gradients  having  any 
structural  similarity  to  the  gradients  relating  to  previously  learned  functions, 
the  networks  that  had  been  trained  with  noise  should  learn  them  faster 
because  they  already  contain  more  information  about  these  gradient  fields 
than  do  networks  whose  adjustments  had  been  made  without  noise. 
Conversely,  if  the  old  and  new  tasks  involved  quite  different  gradients,  the 
ability  of  the  two  types  of  networks  to  learn  the  new  task  should  be  reversed. 
This  occurs  because  the  errors  that  are  generated  during  learning  of  the 
second  task  result  from  the  relative  change  between  the  gradient  fields  of  the 
old  and  new  tasks. 

The  potential  to  store  such  different  types  of  information  may  provide  a 
form  of  generalization  in  biological  systems.  The  first  form  of  encoded 
information  results  from  learning-related  adaptation  to  one  or  more 
conditions.  Given  sufficient  similarity  between  the  new  challenge  and 
previous  ones,  biological  systems  may  gain  considerable  adaptive  advantage 
by  having  the  additional  information  about  the  attractor  gradients. 

CONCLUSION 

The  aim  here  has  been  to  use  attractors  to  make  sense  of  the  functional 
structure  in  large  neural  assemblies.  To  do  this  I  have  discussed  attractors  as 
existing  wholly  within  the  nervous  system.  Learned  perceptions  in  animals 
and  humans  (Freeman  &  Skarda,  1990;  Grossberg,  1988)  and  CPGs  and  are 
examples  of  such  internal  representations.  These  examples  are  equivalent 
because  they  require  sensory  cueing  only  for  triggering  or  sustaining  them, 
although  they  may  incorporate  sensory  or  external  elements  as  part  of  the 
pattern  generating  process  or  modification  of  it  (Kater  &  Rowell,  1973;  Mellon, 
1969).  What  this  says  is  that  if  the  aggregate  activity  of  neurons  in  a  network 
can  not  be  altered  so  as  to  generate  a  particular  basin  of  attraction,  the 
network  will  not  be  able  to  adjust  its  connections  to  encode  the  information 
pertaining  to  that  perception  or  motor  pattern.  Similarly,  a  network  whose 
connections  between  the  neurons  have  already  been  set  will  not  produce  a 


particular  pattern  of  activity  when  driven  by  some  external  signal  if  the  fixed 
points  represented  by  the  connection  stren^hs  do  not  provide  the  gradients 
that  will  express  such  patterns.  In  either  of  these  cases,  the  network  exhibits 
"no  workable  solution"  in  its  ability  to  learn  or  to  respond  to  the  driving  force 
of  an  external  stimulus  once  the  synapses  have  been  set.  Ultimately,  in 
adaptive  behavior,  the  time  varying  neural  output  imderlying  a  behavior 
emerges  from  a  tension  or  dialectic  between  the  goal-directed  needs  of  the 
animal  and  a  shifting  and  unknowable  environment  (Mpitsos  &  Cohan, 
1986a).  The  final  "attractor"  is  not  fixed  hard;  it  is  variable  and  moldable. 

The  conjecture  discussed  here  does  not  unravel  the  specific  connections 
that  we  observe  in  complex  biological  systems  as  those  shown  in  Fig.  1.  It  save 
to  say  that  they  will  never  be  known  in  detail.  We  may  gain  some  assurance  in 
the  validity  of  the  conjecture  by  examining  increasingly  more  complex 
networks  in  simulation  or  testing  the  conjecture  in  biological  networks  in  cell 
culture,  and  showing  that  the  settings  of  the  connections  follow  a  pattern  that 
is  established  by  the  dissipative  dynamics  of  the  system.  But  as  in  Weinberg's 
(1992)  use  of  reductionism,  the  conjecture  provides  the  principle,  not  the 
details,  though  the  effects  of  this  principle  should  be  testable. 
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FOOTNOTE 

iThe  BCNs  are  involved  in  many  different  jaw-mouth  behaviors;  i.e.,  they  are 
multifunctional.  The  animal  exhibits  several  stages  of  feeding.  Like  all  good 
carnivores,  it  actively  exhibits  regurgitation  of  noxious  substances  that  have 
been  ingested.  It  "rejects"  ingested  substances,  by  a  process  that  resembles 
"reverse  play-back"  of  bite-swallow  behavior.  It  exhibits  defensive  biting 
behavior  toward  noxious  attacks.  And,  interestingly,  it  not  only  cleans  its  gill, 
it  also  cleans  the  gill  of  other  Pleurobranchaea,  despite  the  fact  that  the  animals 
are  cannibalistic.  Pleurobranchueu  also  exhibits  rapid  associative  learning  of 
several  forms  using  mouth/head  responses,  eg.,  (Mpitsos  &  Cohan,  1986c; 
Mpitsos  &  Collins,  1975;  Mpitsos,  et  al.,  1978;  Mpitsos,  et  al.,  1988c). 
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FIGURE  LEGENDS 

Figure  1.  "Massively"  parallel  network  connections.  Photomicrograph  of 
immunofluorescence  for  the  neurotransmitter  FMRF-amide  in  the  right  half  of  a 
400-neuron  ganglion  responsible  for  generating  rhythmic  feeding  movements  in 
the  sea  slug,  Pleurobranchaea  californica.  Circular  objects  are  neuron  cell  bodies, 
several  of  which  exhibit  the  green  fluorescence  indicative  of  FMRF-amide.  The 
tangle  of  neurites  in  the  large  area  next  to  the  cell  bodies  is  the  neuropil  where 
most  of  the  synapses  between  neurons  take  place.  Exhaustive  anatomical 
examination  of  the  ganglion  by  slicing  it  layer  by  layer  and  reconstructing  the 
results  shows  that  the  massive  tangle  of  connections  in  the  neuropil  comes  from 
only  a  few  of  the  neurons  in  the  buccal  ganglion  (and  other  ganglia);  i.e.,  that 
there  is  extensive  divergence  in  the  connections  between  neurons.  A  survey  of 
over  a  dozen  neurotransmitters  in  complete  serial  sections  of  all  ganglia  in 
Pleurobranchaea  and  another  sea  slug,  Aplysia,  has  shown  similar  profusely 
diverging  projections  of  a  few  neurons  containing  a  particular  neurotransmitter 
into  vast  areas  of  the  neuropil.  Neurons  containing  other  transmitters  send 
converging  projections  into  the  same  areas.  Physiological  work  confirms  the  high 
degree  of  connectivity  in  Pleurobranchaea  (Mpitsos  &  Cohan,  1986b)  and  Aplysia 
(Zecevic,  et  al.,  1989).  Modified  from  (Soinila  &  Mpitsos,  1991). 

Figure  2.  Biological  networks  compensate  for  the  loss  of  a  neuron.  Traces  show 
the  action  potentials  in  single  neurons  (BCN  and  M)  and  in  nerves  containing  the 
axons  of  many  neurons  (top  four  traces).  The  sequential  biursts  of  action 
potentials  relate  to  feeding  behavior,  where  bursts  in  R1  drive  muscles  that  open 
the  jaws  and  bursts  in  R3  drive  muscles  that  close  the  jaws.  Cessation  of  firing  in 
the  BCN,  caused  by  applying  electrical  current  to  the  BCN  (between  the  arrows), 
stops  the  oscillatory  activity,  but  that  the  oscillations  recover  despite  the  fact  that 
the  BCN  remains  quiescent.  The  BCN  is  one  of  about  20  neurons  that  comprise 
the  oscillator  (CPG)  for  opening  and  closing  the  jaws.  M  is  one  of  the  motor 
neuroris  in  the  mouth  nerve  MN.  The  SOVN  carries  axons  of  nerves  that  control 
muscles  in  the  lips  and  mouth.  Modified  from  (Mpitsos  &  Cohan,  1986b). 

Figure  3.  Dissipative  processes  in  a  computer  simulation  of  periodic  and  chaotic 
changes  in  the  concentration  of  two  of  the  molecular  species  in  a  four-species 
catalytic  network.  Top  four  traces:  time  series  of  the  concentrations  of  molecular 
species  XI  and  X2  before  and  after  a  perturbation  pulse  was  applied  (asterisk). 
The  perturbation  was  applied  by  increasing  the  concentration  of  XI  for  one 
integration  step  and  then  allowing  the  network  to  recover.  Bottom  two 
illustrations:  phase  portraits  of  preperturbed  (thick  lines)  and  perturbed  post¬ 
transient  responses  (thin  lines  starting  at  asterisk)  in  the  two  regimes. 

Figure  4.  Four-dimensional  phase  portrait  showing  relationship  between  four  of 
the  fixed  points  in  periodic  (A  and  B)  and  chaotic  (C)  regimes  and  the  system 
trajectories.  The  concentration  of  molecular  species  XI,  X2,  X4  are  plotted  on  the 
three  axes,  and  X3  is  shown  by  thickness  of  trajectory.  Circles  (spheres)  show  the 


location  of  the  fixed  points.  A:  Unperturbed  system.  B.  Perturbation  induced  by 
injecting  extra  copies  of  XI  for  one  integration  step.  C:  Fixed  points  are  not  shown 
because  they  would  be  obscured  by  the  trajectories.  (A  and  B)  are  modified  from 
(Andrade  et  al.,  1993). 

Figure  5.  Dissipative  action  in  the  simulation  of  small  biologically  realistic 
networks.  The  network  (C)  was  activated  by  steady,  low-level  depolarizing 
current  applied  to  Celll.  A:  Time  series  of  action  potentials  obtained  from  Cell2. 
The  activity  consists  of  a  3-cycle  in  which  three  intervals  between  action 
potentials  repeatedly  appear,  as  shown  by  the  first  seven  action  potentials.  A  125 
msec  hyperpolarizing  perturbation  pulse  was  presented  to  Cell2  (marked  by  the 
dashed  line).  One  action  potential  escaped  during  the  hyperpolarization.  After 
the  release  of  the  hyperpolarizing  pulse,  Cell2  reboimded,  generating  three  action 
potentials  in  rapid  succession,  and  then  relaxed  toward  the  original  3-cycle 
intervals.  B:  Shows  the  return  map  of  the  original  3-cycle  (filled  circles)  and  the 
post-perturbation  recovery  (open  triangles).  The  numbers  in  (B)  are  the  same  as 
in  A,  and  represent  the  interval  between  the  numbered  action  potential  and  the 
action  potential  preceding  it.  C:  Structure  of  the  network:  the  thickness  of  the 
connections  are  proportional  to  the  maximal  synaptic  strengths;  the  numbers 
show  the  strengths  in  milliSiemens.  Excitatory  synapses  (open  circles)  have  a 
reversal  potential  of  45  mV.  Inhibitory  synapses  (filled  circles)  have  a  reversal 
potential  of  -82  mV.  Open  and  close  time  constants  for  both  types  of  synapses  are 
3  and  20  msec,  respectively.  Transmission  delays  were  3  msec.  The  simulations 
were  nm  GENESIS  obtained  from  the  California  Institute  of  Technology, 
Pasadena,  and  used  squid  axon  membrane  (Hodgkin  &  Huxley,  1952). 

Figure  6.  Attractors  may  adjust  synaptic  strengths  globally.  A:  The  same  network 
that  produced  the  3-cycle  activity  shown  in  Fig.  5.  B:  Activity-dependent 
emergence  of  new  connection  strengths:  The  network  was  activated  using  Celll 
to  produce  the  3-cycle.  After  start-up  transients  were  dissipated,  Cell2  was 
stimulated  tonically  in  addition  to  Celll,  and  all  synapses  in  the  network  were 
allowed  to  change  using  a  variant  of  Hebb's  rule.  The  new  synaptic  strengths  are 
indicated  by  the  thickness  of  the  connection  lines  between  the  neurons  and  by  the 
numerical  values  placed  next  to  the  lines  (compare  A  and  B).  C:  A  return  map 
showing  that  the  new  connection  strengths  generate  a  6-cycle  pattern  (open 
circles)  in  the  firing  of  Cell2;  the  original  3-cycle  is  shown  by  the  filled  symbols. 
When  shorter  or  weaker  stimuli  are  presented  to  Cell2,  the  synaptic  strengths  do 
not  settle  sufficiently  into  the  values  required  to  generate  the  6-cycle.  The 
synaptic  strengths  may  fluctuate  as  the  interrelated  firing  patterns  of  the  cells  in 
the  network  seek  one  gradient  or  another.  The  temporal  structure  of  these 
patterns  may  drift  until  a  strong  enough  gradient  emerges  which  then  forces  the 
interrelated  strengths  of  the  synapse  into  some  more  stable  set.  The  activity 
dependent  changes  in  the  synaptic  strengths,  discussed  a  little  later,  used  Mike 
Vanier's  implementation  of  Hebbian  synapses  in  GENESIS. 


Figure  7.  Backpropagation  training  of  an  artificial  neural  network  sets  synaptic 
strengths  optimally  with  respect  to  one  another.  Top:  Schematic  showing  a 
network  having  1  input  neuron,  1  output  neuron  and  n-number  hidden  neurons 
(h).  whi,  and  woi:  synapses  on  hidden  and  output  units,  thi  and  tho,  thresholds  of 
hidden  and  output  units;  here,  i  =  1,..,4.  Bottom:  Training  sets  all  synapses 
optimally  with  respect  to  one  another.  Dashed  vertical  line:  synaptic  strength  set 
during  training.  Curves:  Error  as  a  function  of  changes  in  a  single  synapse  while 
holding  others  at  their  trained  values.  Network  as  in  Fig.  7.  Modified  from 
(Mpitsos  &  Burton,  1992). 

Figure  8.  Two-dimensional  error  surfaces.  Same  procedure  as  in  Fig.  7  except  that 
two  synapses  at  a  time  were  arbitrarily  adjusted.  The  optimum  combination  of 
synaptic  strengths  (shown  by  the  asterisk)  in  the  selected  synapses  (using  all 
other  synapses  at  their  trained  value)  is  identical  to  the  values  obtained  during 
training.  Note  high-error  hill,  nearly  flat  plateaus,  and  the  steeply  sloping 
gradients  of  the  valley  leading  to  the  lowest  error  point. 

Figvure  9.  Error  gradients  in  3-dimensions.  Red  represents  high  error,  and  the 
color  change  from  red  to  blue  shows  decreasing  error.  Same  procedure  as  in  Figs. 
7-8,  except  that  now  the  strength  of  three  synapses  were  adjusted.  A:  Low  error 
resides  somewhere  within  the  block.  B:  Slices  uncover  minimum  error.  The  top 
horizontal  slice  is  identical  to  the  error  surface  shown  in  Fig.  8,  except  that  the 
image  here  is  rotated  -90  degrees.  Non-optimal  slices  show  flat  gradients. 

Figure  10.  High-error  space  grows  exponentially  larger  than  low-error  space  as 
the  number  of  trainable  neurons  increases.  A:  Schematic  of  the  error-surface 
shown  in  the  top  horizontal  slice  of  Fig.  9.  Central  low-error  region  has  been 
made  5  times  larger  by  comparison  to  the  high-error  region  in  the  surround.  B: 
Increasing  to  120  trainable  synapses  reverses  the  relative  sizes  such  that  the 
volume  of  the  surround-space  is  about  100,000  larger  than  the  center  one;  the 
difference  is  a  power  function  given  by  (Ra)^  -  (Rb)^>  where  d  is  the  number  of 
trainable  synapses. 

Figiure  11.  Thresholds  (60  act  as  a  bias  that  shifts  the  sigmoidal  output  functions 
(Output!)  of  hidden  units  selectively  to  different  positions  on  the  activation  axis 
(ai)  which  would  only  be  reached  by  asymptotic  regions  of  output  functions 

when  there  are  no  thresholds  (thick  curve  labeled  No  0's),  ai  is  the  product  of 
the  synaptic  strength  and  the  value  of  the  input  signal,  less  the  threshold  value. 
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Abstract.  The  general  principles  by  which  a  neuron  transforms  the  structure  of 
afferent  synaptic  currents  to  an  output  firing  pattern  are  essentially  unknown.  To 
address  this  problem  we  explored  the  interaction  between  injected  current  and  the 
intrinsic  membrane  conductances  during  the  generation  of  firing  patterns.  This  was 
done  by  performing  perturbation  studies  in  simulations  of  a  simple  model  neuron 
comprising  a  single  compartment  possessing  the  electrical  properties  of  the  well- 
studied  squid  giant  axon  as  originally  described  by  Hodgkin  and  Huxley  [Hodgkin, 

1952  #750].  The  injected  current  consisted  of  delta-function  current  impulses, 
sustained  white  noise,  synaptic  current  or  some  combination  of  these.  Three  sets  of 
experiments  were  performed:  (1)  Subthreshold  white  noise  was  used  to  characterize 
the  impulse-impedance  function  (IRF)  of  the  model  neuron.  (2)  Suprathreshold 
white  noise  was  used  to  obtain  the  average  current  trajectory  (ACT),  which  is  an 
approximation  of  the  first-order  Wiener  kernel  for  action  potential  (spike)  genesis. 

(3)  A  suprathreshold  excitatory  postsynaptic  current  (EPSC)  was  given  as  a  time- 
dependent  source  of  unimodal.  The  EPSC  provided  a  test  condition  for  determining 
how  the  latency  of  spikes  it  evoked  was  influenced  by  the  timing  of  single  current 
impulses.  The  stimulus/response  (i/o)  functions  obtained  from  all  three  methods 
have  complex  multiphased  shapes,  indicating  that  over  short  periods  of  time  linear, 
subthreshold  and  suprathreshold  responses  are  governed  by  the  same  impedance 
process.  These  shapes  also  indicate  that  there  may  be  considerable  degeneracy  in  the 
transfer  of  information  across  synapses.  Unlike  neurons  that  are  coupled 
by  smoothly  graded  synaptic  potentials,  the  dynamics  of  those  relying  on  action- 
potential  evoked  synapses  may  may  not  be  definable  using  systems  of  ordinary 
differential  equations,  though  the  numerical  integrations  generating  the  dynamics 
are  deterministic. 


1  Introduction 


Our  knowledge  about  neurons  falls  roughly  into  two  main  categories.  There  is  a 
private  or  local  world  of  ionic  conductances,  and  a  public  or  non-local  world  of  action 
potentials  and  firing  patterns,  or,  in  the  case  of  nonspiking  neurons,  of  intemeuronal 
communication  that  occurs  via  graded  potentials.  The  fact  that  communication  exists 
between  neurons  implies  that  some  form  of  information  is  processed  locally  and 
transmitted  to  other  neurons  globally.  For  the  moment,  we  use  the  term  "information 
broadly  to  mean  simply  some  definable  characteristic  (or  lack  thereof)  in  the  afferent 
signal  that  a  neuron  receives  and  in  the  output  that  the  local  processes  generate.  This 
characteristic  may  be  as  simple  as  the  mean  firing  frequency  of  an  otherwise  stochastic 
process,  or  it  may  contain  complex  temporal  structure. 

The  local  and  public  worlds  must  be  related  since  the  local  conductance 
mechanisms  transduce  extrinsic  influences  and  support  the  public  events  evoked  by 
them.  The  general  consensus  is  that  the  interactions  among  the  local  events  process 
afferent  information  within  a  neuron  and  export  relevant  information  about  the  afferent 
stream  in  a  efferent  firing  pattern  to  other  neurons  or  to  effector  organs.  Equivalently, 
neurons  that  do  not  generate  action  potentials,  use  graded  potentials  to  broadcast  the 
processed  information  to  other  neurons.  In  the  present  paper  we  begin  the  reports  of 
our  studies  using  findings  obtained  from  model  neurons  that  generate  action 
potentials. 

The  electrophysiology  of  the  single  neuron  stands  at  the  confluence  of  these 
private  and  public  domains.  The  character  of  this  frontier,  the  laws  and  principles 
governing  communication  of  information  between  domains,  is  mostly  obscure  to  our 
understanding  despite  work  and  knowledge  from  both  fields  individually.  Our 
understanding  of  the  basic  mechanisms  of  electrophysiology  come  from  Hodgkin  and 
Huxley  [,  1952  #750]  who  were  the  first  to  present  a  theory  of  excitable  membrane 
potential  based  on  variable  specific  ionic  conductances.  These  conductances  were 
hypothesized  to  result  from  the  presence,  in  a  virtually  impermeable  membrane,  of 
numerous  independent  molecules  capable  of  controlling  the  permeability  of  specific 
ions  through  channels  in  the  membrane. 

These  postulated  channels  possessed  complicated  time-  and  voltage-  sensitive 
characteristics  which  offered  a  good  quantitative  explanation  of  the  action  potential  and 
subthreshold  phenomena.  Subsequent  work  has  essentially  confirmed  the  original 
theory.  Recordings  from  single  channels  has  established  the  relation  between  the 
microscopic  conductances  of  single  channels,  on  the  one  hand,  and  the  macroscopic 
conductances  and  their  control  parameters,  on  the  other  hand,  that  Hodgkin  and 
Huxley  described  [Patlak,  1991  #881].  The  theory  of  specific  ionic  conductances  has 


been  successfully  applied  to  a  wide  range  of  bioelectric  phenomena  such  as  synaptic 
transmission,  sensory  generator  potentials  and  hormonal  influences  on  neural  activity. 

The  study  of  the  public  domain,  of  neuronal  firing  patterns,  goes  back  to  the 
1920s  when  Adrian  [,  1928  #921]  showed  that  changes  in  the  records  from  vertebrate 
peripheral  sensory  nerves  during  stimulation  were  due  to  changes  in  the  firing  rate  of 
single  units  rather  than  to  changes  in  their  shape.  Since  then  it  has  become  clear  that 
firing  rate  alone  is  not  the  only  relevant  parameter  in  neuronal  firing  patterns.  It  has 
been  known  for  a  long  time  that  the  firing  patterns  of  neurons  are  often  not  random  and 
show  significant  structure  [Gerstein,  1964  #916].  Studies  in  visual  systems  suggest  that 
various  types  of  information  about  the  visual  field  can  be  carried  simultaneously  in  the 
firing  patterns  of  single  axons  in  optic  nerve  [Chung,  1970  #871]  and  monkey  visual 
cortex  [McClurkin,  1991  #872].  There  are  numerous  studies  showing  that  the  times  of 
occurrence  of  action  potentials  in  many  neurons  are  not  independent  of  one  another 
[Nakahama,  1977  #873],  and  that  the  degree  of  dependence  varies  with  physiologically 
relevant  variables  [Griineis,  1989  #875;  Yamamoto,  1983  #876]. 

It  is  also  known  that  single  neurons  can  distinguish  patterns  in  the  sequences  of 
synaptic  activity  and  are  potentially  able  to  exploit  this  structure  as  a  source  of 
information.  Random  or  constant-interval  stimulation  completely  changes  the  firing 
pattern  of  pacemaker  neurons  [Perkel,  1964  #878]  and  computer  model  neurons 
[Segundo,  1968  #879].  Stimulating  nerves  with  different  interstimulus  intervals  at  a 
constant  mean  rate  yields  different  firing  patterns  in  identified  Aplysia  neurons 
[Segundo,  1963  #880].  Single  sustaining  fibers  in  crayfish  visual  system  are  capable  of 
carrying  information  relating  to  three  different  types  of  stimuli,  average  illumination, 
spatial  frequency,  and  the  e-vector  of  polarized  light  [Glantz,  1976  #908;  Glantz,  1984 
#910;  Glantz,  1988  #909].  Subtle  changes  in  subcellular  biochemical  processes  are  also 
sensitive  to  the  structure  of  activity  in  the  afferents  to  the  neuron.  For  example,  in  vitro 
and  in  vivo  studies  in  rat,  long-term  potentiation  (LTP)  of  rat  hippocampal  neurons 
exhibits  stimulus-pattern  dependencies  that  may  have  mechanisms  in  common  with  the 
LTP  induced  by  the  standard  procedure  of  equally-spaced  high-  frequency  stimulation 
[Diamond,  1988  #839;  Rose,  1986  #838;  Tsukada,  1994  #917].  Moreover,  simulation 
studies  have  shown  that  dendritic  arrays  are  capable  of  responding  differentially  to 
different  patterns  of  synaptic  input  volleys  [Rail,  1977  #911]. 

Such  studies  indicate  that  neurons  are  able  both  to  recognize  and  to  create 
temporal  structure  in  spike  trains.  These  structures  appear  to  have  precise  relationship 
to  the  structure  of  the  input  signal.  For  example,  information  about  the  movement  of  the 
visual  field  of  the  blowfly  is  recoverable  from  the  structure  of  the  firing  pattern  of  visual 
units  [Bialek,  1991  #870].  The  amount  of  information  that  is  extractable  from  the  firing 
patterns  using  linear  filter  models  can  resolve  movements  smaller  than  the  sensory  field 
of  single  visual  units,  and  approaches  the  noise  level  of  the  optic  detectors.  Recent  work 


on  another  arthropod,  the  crayfish,  lends  further  support  to  the  notion  that  the  firing 
patterns  for  motion  detection  may  be  describable  as  linear  filters  [Glantz,  1994  #919].  It 
would  seem,  therefore,  that  linear  filter  models  of  the  neuron  might  provide  a  bridge  in 
our  understanding  of  the  relationship  between  the  local  computational  properties  of  the 

neuron  and  firing  pattenrs  that  they  generate. 

The  filtering  properties  of  neural  membranes  have  been  known  since  the  late 
1930's,  and  linear  models  of  dynamic  membrane  conductances  [Cole,  1968  #920;  Koch, 

1984  #922;  Mauro,  1970  #882;  Sabah,  1969  #924;  Stein,  1972  #925]  have  established  the 
relationship  between  the  conventional  Hodgkin-Huxley  theory  of  specific  ionic 
conductances  and  complex  impedances  of  neurons  and  other  excitable  cells.  However, 
the  systems  of  conductances  supporting  bioelectric  phenomena  in  neurons  are 
unambiguously  nonlinear.  Linear  models  are  only  applicable  over  narrow  ranges  of 
membrane  potentials,  usually  well  below  threshold  (e.g.,  see  [Koch,  1984  #922]), 
whereas  much  larger  ranges  of  voltage  excursions  appear  to  be  important  or  necessary 
for  information  processing  in  spiking  neurons.  For  this  reason,  linear  filter  models  are 
not  often  considered  generally  useful  for  describing  spike  generation  or  single  neuron 
computation.  Nonlinear  models  have  been  developed  to  account  more  accurately  for 
the  structure  in  spike  trains  of  biological  systems,  e.g.,  [French,  1989  #888]. 

Nevertheless,  the  empirical  success  of  linear  filter  models  to  account  for  the 
input/output  relations  in  many  neurons  [Bryant,  1976  #815;  Enroth-Cugell,  1983  #972; 
Frishman,  1987  #973;  Krause,  1980  #971;  Sakuranaga,  1985  #844;  Sakuranaga,  1985  #843; 
Sakuranaga,  1983  #845],  and  the  linear-filter  decoding  of  the  spike  trains  of  motion 
detectors  in  the  blow  fly  visual  system  [Bialek,  1991  #870],  suggest  that  linear  filter 
models  may  be  more  generally  applicable  than  is  usually  believed.  The  presence  of 
high-precision  information  in  the  temporal  structure  of  firing  patterns  in  the  blowfly 
motion  detectors  suggests  the  presence  of  robust  membrane  mechanisms  that  preserve 
and  respond  to  small  changes  in  structure  of  input  signals  [Bialek,  1991  #870;  Bialek, 
1993  #915].  Although  questions  of  the  effects  of  nonlinearities  must  be  addressed,  the 
work  of  Guttman,  Feldman  and  Lecar  [,  1974  #914]  on  squid  axon  membrane  suggests 
that  violations  of  linearity  produce  only  weak  in  the  filter  properties.  The  linear 
approximations  to  nonlinear  activity  in  cockroach  tactile  sensory  hairs  [French,  1989 
#888]  suggest  the  same  conclusion.  Linear  analytical  methods  often  provide  excellent 
characterizations  of  invariant  measures  of  the  nonlinear  dynamics  generated  by  systems 
of  coupled  differential  equations,  e.g.,  [Andrade,  1993  #635;  Canavier,  1990  #517,  Wolf, 

1985  #55].  Taken  together  such  studies  suggest  that  it  is  reasonable  to  revisit  the  subject 
of  how  linear  filter  models  of  membrane  mechanisms  may  be  involved  in  the 
computational  functions  of  neurons. 

In  the  series  of  studies  reported  here  we  inquire  into  plausible  explanations  of 
how  subthreshold  linear  filter  interactions  in  neuron  membrane  can  govern  the 


structure  of  nonlmear  processes  comprising  spike  trains.  We  show  how  a  minimally 
complex  membrane  model  can  both  preserve  and  transform  temporal  information.  We 
suggest  that  these  preserving  and  transforming  effects  may  be  largely  explained  by  a 
linear  filter  model  operating  near  a  threshold  nonlinearity.  We  then  introduce  some 
basic  consequences  of  the  insights  concerning  single  neuron  computation  which  will  be 
considered  more  fully  in  later  publications. 

2.  Methods 

2.1  The  simulation  neuron 

All  of  the  computer  simulations  in  this  study  used  SWIM,  version  1.5,  a  simulator 
for  real  neural  networks  (Ekeberg,  et  al.,  1990),  obtained  from  the  Department  of 
Numerical  Analysis  and  Computing  Science,  Royal  Institute  of  Technology,  Stockholm, 
Sweden.  The  integrations  employed  the  Crank-Nicholson  method.  The  size  of  the 
integration  steps  ranged  between  1  and  100  |isec,  depending  on  the  purpose  of  the 
experiment.  The  SWIM  source  was  compiled  with  GNU  gcc  version  2.4.8  (Free  Software 
Foundation  Inc.)  using  IEEE  standard  compliant  double  precision  floating  point 
arithmetic  libraries. 

The  SWIM  source  was  modified  to  accept  novel  current  sources  that  were  not 
part  of  the  original  program.  The  modifications  dealt  strictly  with  adding  novel  current 
sources;  none  of  the  integration  or  other  code  was  altered.  These  modification  include 
the  ability  to  read  arbitrary  current  waveforms  from  script  files,  a  white  noise  current 
source,  transient  current  pulses  lasting  single  time  steps,  synaptic  conductances  with  bi¬ 
exponential  time  courses  and  ionic  conductances  using  the  Hodgkin  and  Huxley 
[Hodgkin,  1952  #750]  specifications  for  squid  giant  axon.  It  was  also  modified  to 
perform  event  triggered  signal  averaging  used  in  the  ACT  experiment.  A  shifting  buffer 
containing  the  recent  history  of  injected  current  was  maintained.  When  an  action 
potential  occurred  the  contents  of  the  buffer  were  used  to  update  a  buffer  of  sums  and 
sum  of  squares  of  the  recent  current  history. 

In  all  cases  we  modeled  a  simple  spherical  neuron  comprising  a  single 
compartment  with  no  axons,  dendrites  or  other  added  complexities.  We  used  three 
simulation  models  which  differ  only  in  the  types  of  current  used  to  drive  the  modeled 
neuron.  In  one,  white  current  noise  was  used.  In  the  second,  a  generic  simulated 
presynaptic  neuron  was  used  to  generate  an  EPSC  in  conjunction  with  a  single  current 
pulse,  lasting  one  integration  step,  located  at  various  times  in  the  simulation  epoch.  In 
the  third  case  the  neuron  was  driven  by  current  waveforms  obtained  from  edited 
portions  of  the  ACT,  obtained  from  a  previous  white  noise  simulation,  that  were  read  as 
a  current  command  script. 


The  neuronal  membrane  parameters,  described  below,  are  taken  verbatim  from 
the  description  of  the  squid  giant  axon  at  6.3  C®  by  Hodgkin  and  Huxley  [Hodgkin, 
1952  #750].  The  parameters  listed  below  were  normalized  to  take  into  account  the  size 
of  the  neuron. 

Model  Neuron  Parameters: 

a  i  Constant  Parameters 

Radius  of  Neuron:  44  x  10"^  m;  Area  =  6.082  x  10"^  m^ 

Membrane  capacitance:  Cm  =  0.01  Farads/m^ 

Maximum  conductances:  ~  1200  S/m^  ;  =  360  S/m^;  gieak  ~  ^  S/m^ 


Reversal  potentials;  Ejsja  =  0.055  V;  Ek  =  ■  0.072  V;  Ej^eak  -  -0.0604V 


bi  Voltage-Dependent  Characteristics  of  the  Squid  Giant  Axon 

dV_  ^syn 

IF- 

+ gK(V-EK> + Sl(v-e^ + /, + Is,.]  (2) 

^m^hg,sJ.V-E„,)  +  n^gK(V-E,)  +  gdV-E^  +  h  +  /^„]  (3) 

^=a„,(l-m)-pjn  (4) 


^=ahil-h)-phh 

^=a„(l-n)-P„n  (6) 


ri  Steady  State  Resting  Parameter  Values 


m  =  0.0355402 

n  =  0.260777 

h  =  0.705487 

Em  = -0.0631V. 

The  functions  and  parameters  used  to  obtain  the  rate  constants  (ocm,  fim,  cch,  Ph> 
ctn,  Pn)  for  opening  and  closing  of  the  ion  gates  {m,  h,  n)  were  identical  to  those 
described  by  Hodgkin  and  Huxley  [Hodgkin,  1952  #750]. 

2.2  Synaptic  conductances 

The  time  course  for  the  EPSC  was  modeled  to  roughly  resemble  the  squid  giant 
synapse,  as  described  by  Llinas,  Steinberg,  and  Walton  [Llinas,  1980  #831].  With  the 
parameters  given  below,  the  model  synaptic  conductance  reaches  a  peak  of 
approximately  25%  of  its  maximum  value  (gsyn)  after  about  0.5  msec.  The  maximum 
synaptic  conductance  was  chosen  by  trial-and-error  to  be  just  superthreshold  at  the 
normal  resting  potential. 

EPSC  Parameters: 

Reversal  potential  (Esyn):  0  volts. 

Maximum  conductance  (gsyn):  3.8xl0"8  Siemens. 

Delay  after  presynaptic  action  potential:  variable. 

EPSC  activation  time  constant  (xi);  0.00035  sec. 

EPSC  inactivation  time  constant  (t2):  0.0007  sec. 

The  time  course  of  the  synaptic  current  is  by  the  following  equations,  where  V  is 
the  time-dependent  membrane  voltage: 

Isy.  =  gsyn(.V-E^.„)  (7) 

gsyn  =  gs,:XO-e-’l^^)-(l-e-<lH)\  (8) 

2.3  Average  current  trajectory  (ACT) 

Use  of  the  ACT  asks  the  questions,  given  the  occurrence  of  a  sequence  of  action 
potentials,  what  can  we  predict  of  the  history  of  the  applied  current  that  lead  to  these 
action  potentials?  One  could  ask  the  opposite  question,  given  the  history  of  the  applied 
current,  what  can  we  predict  of  firing  of  a  neuron?  For  the  present  paper,  we  restrict  the 
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presentation  to  the  first  question.  The  term  "trajectory"  refers  to  the  time  course  of  the 
shape  of  the  average  membrane  current  that  precedes  the  generation  of  action 
potentials.  The  methods  used  here  closely  follow  those  of  Bryant  and  Segundo  [Bryant, 
1976  #815]  who  examined  the  ACT  in  identified  neurons  of  Aplysia. 

To  obtain  the  ACT,  low-level  Gaussian  white-noise  current  X(t)  was  injected  into 
a  neuron  at  each  integration  step,  and  only  one  neuron  was  used;  i.e.,  it  was  not 
synaptically  activated.  The  response  Y(t)  of  the  membrane  to  the  applied  current  was  a 
function  only  of  the  injected  current  X(t).  Y(t)  was  converted  to  a  delta  function 
representing  the  sequence  of  action  potentials  that  were  generated  over  time,  by  the 
following  algorithm: 


i  =  1 


L  0,  otherwise, 


(9) 


(10) 


where  ti  represents  the  time  of  occurrence  of  action  potentials  which  were  detected 
using  a  threshold  of  -30  mV  in  the  membrane  potential. 

The  ACT  was  calculated  using  an  averaging  function  S(t)  that  correlates  the 
occurrence  of  action  potentials  with  the  membrane  current  that  precedes  them: 


(11) 

N  represents  the  total  number  of  action  potentials  in  the  recorded  spike  train,  ti 
represents  the  time  at  which  each  action  potential  occurs,  and  xi  is  the  time  preceding 
the  action  potentials.  Here  Tj  =  At,  for;  =  1,...,M,  where  At  is  the  integration  step  size. 

The  total  observation  time  T  before  the  action  potentials  is  defined  by  T  =  MAt. 

To  compute  the  ACT,  S(r)  is  evaluated  for  all  N  action  potentials  by  summing  all 
of  the  currents  injected  at  each  Tj  Therefore,  the  ACT  shows  the  average  current  injected 
at  a  series  of  M  points  before  the  appearance  of  an  action  potential.  Thus,  the  ACT  is 
both  a  normalizing  and  an  averaging  process.  It  scans  all  records  for  the  occurrence  of 
action  potentials,  sets  that  as  time  "zero"  and  then  examines  the  average  current  across 
the  membrane  for  a  series  of  times  preceding  the  action  potential. 

Depending  on  the  length  of  time  one  uses  to  obtain  the  ACT  and  the  firing 
frequency,  one  action  potential  can  interfere  with  another.  To  avoid  such  artifacts,  we 
selected  the  noise  variance  of  the  injected  current  such  that  the  interval  between  action 
potentials  was  usually  greater  than  50  msec,  the  interval  used  to  calculate  the  ACT. 


Action  potentials  were  excluded  from  the  analysis  that  were  preceded  by  another  action 
potential  by  less  than  70  msec. 


2.4  Complex  impedance  (Z)  and  the  impulse  response  function  (IRF) 


The  complex  impedance  Z  is  the  frequency  domain  image  of  the  ratio  of  the 

complex  frequency  domain  image  of  voltages  with  respect  to  the  complex  frequency 

FFTiV(t)] 

domain  image  of  the  applied  current;  Z  =  FFT  is  the  fast  Fourier  transform. 

V(t)  is  the  membrane  potential  time  series  calculated  by  SWIM  in  response  to  the 
applied  white-noise  current  I(t),  which  is  described  below,  both  of  which  are  in  the  time 
domain.  Signal  processing  routines  were  taken  from  Stearns  and  David  [Steams,  1993 
#864].  Our  methods  follow  closely  after  Puil,  Gimbarzevsky,  and  Miura 
[Gimbarzevskey,  1986  #865].  The  reported  impedances  are  averages  of  ten  60  msec 
"sweeps",  each  sweep  consisting  of  a  different  simulation  run.  A  different  set  of  random 
signals  was  used  in  each  simulation  sweep. 

The  IRF  is  obtained  from  Z  by  simply  calculating  the  inverse  Fourier  transform 
to  transpose  the  complex  impedance  vector  to  the  time  domain.  These  white-noise 
injection  methods  were  the  same  as  those  used  to  obtain  the  data  used  for  the  ACT, 
except  that  the  mean  noise  level  was  kept  low  enough  to  prevent  the  generation  of 
action  potentials.  A  second  difference  is  that  while  the  data  for  the  ACT  was  obtained 
using  continuous  simulations  to  obtain  tens  of  thousands  of  action  potentials,  the 
method  for  obtaining  the  data  for  the  IRF  used  short  60  msec  sweeps. 


2.4  Random  current 

The  white  noise  was  generated  using  the  ranO  and  gasdev  functions  from  [Press, 
1988  #836],  pp.  207-208,  which  uses  the  algorithm  of  Bays  and  Durham  described  in 
[Knuth,  1981  #835].  The  random  number  generator  was  seeded  with  the  current  clock 
time  when  the  program  was  rim  to  avoid  repeatedly  using  the  same  random  series.  This 
algorithm  produces  a  normally  distributed  deviate  with  zero  mean  and  unit  variance. 
SWIM  was  modified  so  that  at  each  integration  step  of  the  simulation,  a  new  current 
intensity  was  selected  from  a  Gaussian  distribution  with  an  appropriately  specified 
variance.  Different  variances  were  used  under  different  circumstances  to  control  the 
range  of  the  current  signal.  A  variance  of  2xl0"12  amps  was  used  for  estimating  the 
membrane  impedance  and  0.9  xlO"^  amps  was  used  in  the  ACT  experiment.  The  former 
produced  strictly  subthreshold  noise  and  the  latter  produced  occasional  action 
potentials. 


The  membrane  of  the  simulated  neuron  acts  as  a  low  pass  filter  and  its  transfer 
function  falls  rapidly  between  10  and  100  Hz.  The  components  of  the  stimuli  with 
frequencies  above  100  Hz  have  negligible  effect  on  the  membrane  potential  but  are 
susceptible  to  phase  distortions.  To  reduce  this  source  of  noise  and  restrict  the  signal 
power  to  a  relevant  range  of  frequencies  in  the  experiments  estimating  the  membrane 
impedance,  we  used  a  three-stage  Butterworth  low-pass  filter  [Steams,  1993  #864]  with 
a  cutoff  frequency  of  400  Hz.  The  white-noise  current  used  to  obtain  the  ACT  were 
filtered  using  a  seven-point  centered  weighted  ruiming  average  to  reduce  large 
transients  which  made  averaging  difficult.  Since  the  ACT  was  obtained  using 
integration  steps  of  0.050  msec  or  less,  whatever  correlations  the  filtering  produced 
were  maintained  within  short  time  spans  of  only  about  0.350  msec.  In  both  cases  blocks 
of  random  numbers  were  calculated  and  filtered  in  chunks  to  provide  consistent  and 
smooth  filtering  properties  over  the  course  of  each  simulation  run. 

3.  Results 

We  are  interested  in  understanding  how  current  signals  affect  the  activity  of 
neurons.  In  a  general  sense,  the  effects  produced  by  a  perturbing  current  signal  depend 
on  two  independent  factors.  The  structure  of  the  current  (the  current  amplitude  over 
time)  and  the  electrical  characteristics  of  the  membrane. 

The  structure  of  the  current  is  largely  determined  non-locally  by  the  network  in 
which  the  neuron  is  embedded  or  by  the  nature  of  the  pulses  that  are  applied 
experimentally.  However,  the  local  membrane  characteristics,  which  are  themselves 
complexly  varying  dynamic  products  of  history,  determine  how  the  structure  of  the 
current  is  processed  by  a  neuron  at  any  given  time.  It  is  therefore  necessary  to  consider 
the  two  factors  together  as  a  single  functional  unit. 

We  choose  to  restrict  our  attention  here  to  stable  cells  at  rest  with  no  sustained 
activity  resulting  from  intrinsic  pacemaker  conductances,  conspicuous  bursting 
tendencies  or  constant  superthreshold  depolarizing  current.  Neither  do  we  concern 
ourselves  here  with  overwhelmingly  large  synaptic  currents.  Instead,  we  look  at  the 
interactions  of  combinations  of  simple,  transient  currents  with  small  to  intermediate 
magnitudes  in  otherwise  quiescent  neurons  where  the  sole  source  of  afferent 
information  is  contained  in  the  structure  of  the  current  signal. 

It  is  first  necessary  to  appreciate  the  degree  to  which  different  components  of  a 
current  signal  can  be  treated  imequally.  Synaptic  interactions  are  often  informally 
referred  to  as  "spatial  and  temporal  summation."  However,  the  membrane  is  a  variable 
and  complex  electronic  filter  and  the  relation  between  current  and  voltage  is 
substantially  more  complex  than  this  suggests  and  involves  frequency-dependent 


distortions  of  signal  amplitude  and  phase.  This  relation  is  described  by  the  membrane 
impedance. 

3.1  Membrane  Impedance 

Impedance  relates  voltage  to  applied  current  in  the  same  way  that  resistance 
relates  voltage  to  current  in  Ohm's  law  except  that  while  Ohm's  law  applies  only  to 
circuits  comprising  constant,  frequency-insensitive  components  (resistance),  impedance 
also  describes  the  voltage  distribution  across  and  current  flow  among  frequency- 
sensitive  components  (capacitance  and  inductance).  The  effects  of  applied  currents  and 
their  combinations  on  membrane  potential  cannot  be  understood  without  knowing  the 
impedance  of  the  neuron.  As  a  basis  for  subsequent  studies,  we  start  with  an 
examination  of  the  impedance  characteristics  of  the  Hodgkin-Huxley  model  under  the 
conditions  that  we  shall  place  it  under  the  various  perturbation  experiments. 

In  order  to  measure  the  impedance  of  the  model  neuron  it  is  necessary  to 
measure  the  impedance  at  many  different  frequencies  since  any  circuit  which  contains 
reactive  components,  such  as  a  neuron  membrane,  will  have  different  impedances  at 
different  frequencies.  Because  impedance  is  a  function  of  frequency  and  not  a  single 
number  like  resistance,  this  analysis  is  typically  performed  in  the  frequency  domain 
where  impedance  is  a  complex  quantity.  Resistance,  the  component  of  impedance 
which  does  not  change  in  response  to  changes  in  current,  is  the  real  part  and  reactance, 
the  component  of  impedance  which  reacts  to  changes  in  current,  is  the  imaginary  part. 
Impedance  is  usually  presented  as  impedance-plane  plots  (Fig.  1  A  and  C),  where 
resistance  is  plotted  against  reactance  at  all  frequencies,  and  as  individual  magnitude 
(Fig.  IB)  and  phase  angle  (Fig.  ID)  vs.  frequency  plots.  The  two  representations  are 
related  by  virtue  of  the  fact  that  each  point  on  the  impedance  plane  can  be  represented 
as  a  vector  drawn  from  the  origin  to  the  point;  its  length  is  impedance  magnitude,  and 
its  angle  is  the  phase  angle  of  the  impedance.  The  phase  angle  can  be  positive  or 
negative  and  serves  to  distinguish  the  two  classes  of  reactance  depending  on  whether 
its  phase  angle  is  positive  (inductive)  or  negative  (capacitive). 

The  two  types  of  graphs  complement  each  other.  The  points  along  the  curves  on 
the  impedance  plane  are  ordered  according  to  frequency  with  the  highest  frequencies 
near  the  origin.  Progressively  lower  frequency  impedances  are  encountered  by 
following  the  curve  counterclockwise  from  the  origin  imtil  the  other  end  is  reached 
which  corresponds  to  the  zero  frequency  or  DC  resistance  of  the  membrane.  This  shows 
nicely  how  impedance  changes  smoothly  and  continuously  over  a  range  of  frequencies 
but  provides  no  exact  information  about  those  frequencies.  That  information  is 
provided  by  the  other  graphs  where  frequency  is  the  independent  variable  and  can  be 
read  directly. 
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In  practice  impedance  is  measured  by  injecting  the  neuron  with  some  time 

varying  current  signal,  collecting  the  voltage  response  of  the  neuron  to  it  and  then 

_  FFT{V{t))  ,  .  , 

calculating  the  impedance  according  to  the  equation  Z  -  where  Z  is  me 

complex  impedance,  FFTO  is  the  fast  Fourier  transform  function  (SPFFTRO)  in  Steams 
and  David,  1988)  which  converts  a  time  domain  vector  into  its  complex  frequency 
domain  image,  V(t)  is  the  voltage  time  series  observed  in  response  to  I(t),  the  current 
stimulus  time  series.  White  noise  is  used  here  as  the  stimulus  source  because  it  is 
convenient  and  contains  nearly  equal  signal  power  at  all  frequencies. 

At  least  ideally.  Any  single  finite  sample  of  white  noise  will  be  incomplete  with 
unequal  power  at  different  frequencies.  To  mitigate  this  finite  sampling  artifact  the 
results  presented  here  are  the  averages  of  ten  independent  estimations  of  Z.  Examples 
of  estimates  of  the  model  neuron  impedance  are  shown  in  Fig.  1.  They  were  obtained 
under  three  different  conditions:  normal  resting  membrane  potential  with  0.1  msec 
model  time  steps,  normal  resting  membrane  potential  with  0.05  msec  model  time  steps, 
and  hyperpolarized  resting  membrane  potential  with  0.05  msec  model  time  steps. 

FIGURE  1  HERE 

At  its  normal  resting  potential  the  membrane  acts  as  a  pure  resistor  only  at  the 
two  frequencies  0  and  20  Hz,  indicated  by  the  zero  phase  angle  axis  crossings  in  Fig.  ID. 
These  points  correspond  to  the  two  intersections  of  the  resistance  axis  in  the  impedance 
plane  plots  (lA  and  1C).  Changing  the  polarization  of  the  membrane  significantly  alters 
its  impedance.  This  is  apparent  when  comparing  the  impedance  plane  plots  of  the  data 
from  normal,  depolarized  and  hyperpolarized  membranes  in  Figure  lA,  where  the 
hyperpolarized  membrane  exhibits  only  reactive  impedance  except  at  0  Hz.  At  the 
normal  resting  membrane  potential  the  magnitude  of  the  reactance  rises  from  its  DC 
value  to  peak  at  45  Hz  (Figure  IB)  and  then  falls  off  rapidly.  Depolarizing  the  ceU  as 
little  as  3  mV  significantly  decreases  the  low  frequency  impedance,  sharpens  the 
impedance  peak  and  shifts  it  up  to  55  Hz  (Fig.  IB)  Hyperpolarizing  the  cell  as  little  as  3 
mV  noticeably  the  low  frequency  impedance,  broadens  the  impedance  peak  and  shifts  it 
down  to  about  30  Hz.  Above  100  Hz  the  three  impedance  curves  are  similar. 

Peaked  impedance  curves  such  as  those  in  Figure  IB  are  characteristic  of  a 
damped  oscillatory  circuit  which  is  indicative  of  combined  inductive  and  capacitive 
elements,  e.g,  see  [Mauro,  1970  #882;  Sabah,  1970  #928;  Sabah,  1969  #924].  The  reactance 
of  the  normal  membrane  is  largely  inductive  at  and  below  20  Hz,  as  shown  by  the 
positive  reactance  in  Figure  ID.  Three  mV  depolarization  shifts  the  crossover  point  to  40 
Hz.  At  higher  frequencies  and  increasing  polarization,  the  reactance  is  increasingly 
dominated  by  capacitance,  as  shown  by  the  negative  swing  of  the  phase  (Fig.  ID).  A 


pure  RC  circuit  will  exhibit  a  semicircle  in  an  impedance  plane  plot.  The  deviations 
from  this  form  in  lA  and  1C  show  that  even  though  the  net  reactance  is  capacitive,  there 
is  a  smaller  but  still  significant  inductive  component  at  the  hyperpolarized  potential. 

As  the  frequency  of  the  stimulus  signal  increases  capacitance  shunts  the  current, 
and  the  voltage  response  falls  off  rapidly  above  45  Hz,  the  resonant  impedance  peak  at 
rest  (Fig.  1).  At  normal  and  depolarized  resting  potentials,  and  at  low  frequency 
stimulation,  the  membrane  impedance  is  significantly  influenced  by  inductive  reactance 
arising  from  the  dynamics  of  the  sodium  and  potassium  conductances. 

Within  the  modeling  parameters  used  in  this  study  the  integration  time  step  size 
does  not  introduce  artifacts  at  functionally  relevant  frequencies.  It  can  be  seen  in  Fig.  1 
that  the  only  effect  of  changing  the  time  step  from  0.050  to  0.100  msec  is  to  introduce 
some  phase  distortions  at  frequencies  above  300  Hz  while  most  of  the  voltage  signal 
power  is  below  200  Hz.  In  the  remaining  experiments  time  steps  of  0.050  msec  or  less 
will  be  used. 

3.2  The  Impulse  Response  Function  (IRF) 

The  IRF  is  the  time  domain  image  of  the  impedance  and  shows  a  three-phase 
time  course  of  impedance  change  that  would  occur  in  the  membrane  in  response  to  brief 
small  current  impulses.  The  significance  of  the  IRF  stems  from  the  fact  that  the  voltage 
response  to  any  current  signal  can  be  calculated  by  convolution  with  the  IRF.  For  a 
single  current  impulse  the  voltage  response  will  be  the  IRF  times  the  amplitude  of  the 
current  impulse.  So  the  IRF  indicates  how  any  current  signal  will  be  transformed  and 
drawn  out  in  time. 

It  is  obtained  by  calculating  the  inverse  Fourier  transformation  of  Z  (SPIFTRQ)  in 
Stearns  and  David,  1988).  The  IRF  from  the  two  experiments  with  0.05  msec  time  steps 
discussed  above  are  shown  in  Figure  2.  The  two  curves  show  clearly  why  inductive 
filters  are  considered  as  damped  oscillators.  In  both  cases  a  single  current  impulse 
causes  an  oscillation  in  the  membrane  impedance  which  diminishes  over  time.  The 
membrane  potential  shows  the  same  damped  oscillation  in  response  to  a  small  current 
impulse  (Figure  9  A  and  C).  The  depth  of  the  negative  impedance  trough  decreases  with 
hyperpolarization  and  later  than  at  the  normal  resting  potential. 

FIGURE  2  NEAR  HERE 

In  an  ideal  linear  filter  these  damped  oscillations  will  continue  indefinitely.  In 
these  simulations  the  ability  to  detect  them  is  limited.  The  IRFs  merge  mto  ambient 
noise  after  20  to  30  ms.  The  important  observation  about  the  IRF,  which  we  shall 
examine  further  in  the  ACT,  is  that  information  about  extremely  small  perturbations 


(maximum  amplitude  of  only  about  1.5  x  10-12  Amps  producing  maximum  voltage 
excursions  of  only  0.025  mV  (Figure  1  legend))  is  retained  and  still  detectable  over  many 
milliseconds. 

3.3  The  Average  Current  Trajectory  (ACT):  Using  the  output  stream  as  the  frame  of  reference 

The  relevance  of  the  membrane  impedance,  as  it  was  described  above,  to  the 
behavior  of  the  membrane  assumes  that  the  membrane  acts  as  a  linear  filter.  This 
means  that  the  state  of  the  membrane  at  any  given  time  is  the  sum  of  the  effects  of  prior 
events.  The  assumption  is  only  valid  for  a  small  range  of  potentials  within  about  5  mV 
of  resting  in  scjuid  axon  [Koch,  1984  #922;  Sabah,  1969  #924;  Mauro,  1970  #882].  The 
current  pulses  we  used  were  well  within  this  range,  and  far  from  threshold,  since  the 
maximum  voltage  excursion  to  obtain  the  IRF  was  0.025  mV  above  the  -63  mV  resting 
potential,  approximately  10  mV  below  the  -54  mV  firing  threshold. 

The  membrane  conductances  in  general,  especially  near  threshold,  comprise  a 
nordinear  system,  and  the  steady-state  impedance  over  the  range  of  voltages  through 
which  the  membrane  potential  must  journey  in  order  to  achieve  threshold  was  shown  to 
change  significantly.  One  should  not  expect  that  the  IRF,  which  is  obtained  near  the 
resting  potential,  will  be  a  reliable  predictor  of  the  behavior  of  the  membrane  over  the 
range  of  voltages  up  to  its  firing  threshold.  Therefore,  inasmuch  as  we  are  interested  in 
the  relation  between  current  and  the  timing  of  action  potentials,  a  means  of  ascertaining 
how  signals  will  interact  and  combine  over  this  range  is  needed.  The  ACT  described  in 
this  section  correlates  the  occurrence  of  action  potentials  with  the  recent  history  of 
current  perturbations.  This  provides  a  direct  empirical  measure  of  the  relation  between 
the  current  and  the  spike  generating  mechanism  which  can  be  compared  to  the  IRF. 

As  in  the  case  of  the  IRF,  the  neuron  was  activated  by  applying  randomly 
varying  current.  With  the  parameters  used  here  the  cell  exhibited  an  overall  firing  rate 
of  16  spikes  per  second  (45,781  spikes  in  2,849.1  seconds).  The  ACT  was  acquired  from 
30,000  spikes  (65.5%).  The  remaining  34.5%  were  rejected  because  they  were  less  than  50 
msec  apart.  Fig.  3  shows  an  excerpt  of  a  train  of  action  potentials  obtained  in  a  sample 
simulation  run.  The  ACT  is  shown  in  Fig.  4. 

FIGURES  3  &  4  HERE 

There  are  three  statistically  significant  phases  in  the  ACT  time  course,  noted 
numerically,  showing  a  damped  oscillation.  The  ACT  is  an  average  correlation  function 
representing  the  average  or  most  likely  current  trajectory  "chosen"  by  the  neuron  from 
the  current  waveforms  presented  to  it.  Edited  portions  of  the  ACT  were  used  as 
stimulus  waveforms  to  ascertain  the  functional  significance  of  the  three  phases.  It  can 


be  seen  from  Fig.  5  that  the  early  positive  phase-3  produces  no  change  in  the  voltage 
response  that  is  observable  at  this  resolution.  The  middle  negative  phase-2  is  not 
necessary  to  evoke  an  action  potential,  but  it  does  accelerate  the  rise  of  the  action 
potential  and  reduce  its  latency.  The  late  positive  phase-1  alone  is  sufficient  to  produce 
an  action  potential,  but  the  latency  is  prolonged  with  respect  to  the  complete  ACT 
stimulus  waveform.  The  ACT  therefore  is  not  a  minimal  threshold  current  just 
adequate  to  evoke  an  action  potential.  It  is  a  series  of  synergistic  components  which 
support  each  other.  The  phases  of  the  ACT  are  temporally  correlated  with  the  three 
phases  of  the  IRF  which  is  a  subthreshold  function. 

FIGURE  5  NEAR  HERE 

3.4  Modified- ACT:  Combining  sSbthreshold  Random  Noise  with  a  Spike-Evoking  EPSC  to 

Allow  Changes  in  the  Frame  of  Reference  in  Assessing  Spike  Trains 

3.4.1  Information  obtained  using  modified- ACTs  from  the  reference  of  the  output 

With  the  ACT,  noise  was  the  sole  driving  influence.  In  order  to  understand  how 
impedance  responds  to  combinations  of  current  that  might  be  encountered  by  a  neuron 
in  a  circuit,  we  added  a  moderately  strong  synaptic  event  to  represent  a  deterministic 
aspect  of  the  network's  business.  It  is  deterministic  because  of  the  constancy  of  its  time 
of  presentation,  amplitude,  and  shape.  The  aim  is  to  influence  this  suprathreshold, 
deterministic  process  with  subthreshold  random  stimulation. 

Since  there  are  now  two  independent  stimulus  sources,  the  noise  and  the 
synaptic  current,  there  are  two  ACTs;  one  of  the  average  current  trajectory  of  the  noise 
channel,  and  one  of  the  synaptic  current  channel.  Although  the  complete  stimulus 
waveform  is  the  sum  these  currents,  we  shall  treat  them  separately  here  because  they 
provide  independent  sources  of  information  with  which  to  analyze  the  response  of  the 
membrane.  Moreover,  because  the  noise  is  so  much  smaller  in  amplitude  than  the 
synaptic  current,  any  details  about  the  noise  source,  and  of  the  relationship  between  it 
and  its  affect  on  the  membrane,  would  be  invisible  if  only  the  sum  were  observed.  We 
shall  refer  to  this  subthreshold  noise  ACT  as  the  modified- ACT  to  distinguish  it  from 
the  standard  ACT  in  which  the  noise  is  suprathreshold,  and  which  did  not  involve  the 
activation  of  the  synapse.  The  synaptic  component  of  the  current  trajectory  will  be 
referred  to  as  the  synaptic- ACT. 

In  these  experiments,  35  ms  epochs  of  time,  were  simulated  rather  than  a  single 
long  simulation  as  used  for  the  ACT  above.  The  noise  was  entirely  subthreshold.  In 
each  simulation  an  EPSC  of  constant  magnitude  was  presented  at  a  constant  delay  into 
the  simulation.  In  the  absence  of  noise,  the  EPSC  evoked  an  action  potential  at  30.96  ms. 


With  the  addition  of  noise,  the  latency  of  the  action  potential  varied  since  each  35  ms 
sweep  of  the  simulation  used  a  different  set  of  randomly  generated  noise  pulses  in 
conjunction  with  the  EPSC.  Only  one  or  action  potential  was  obtained  in  an  epoch 
(sweep);  in  a  few  cases  the  action  potential  failed.  Two  intensities  of  noise  were  used  to 
examine  the  modified  and  synaptic  ACTs  under  different  membrane  conditions.  With 
the  lower  noise  intensity,  82,503  simulation  sweeps  were  conducted  in  which  there  were 
9326  action  potential  failures  (0.75%).  We  refer  to  these  failures  as  class  F  "latencies". 
With  the  higher  noise  intensity,  82,503  sweeps  were  run  in  which  there  were  9,326 
failures  (11.3%).  In  both  experiments,  60,000  action  potential  latencies  were  used  to 
compute  the  modified  and  synaptic  ACTs.  The  method  was  the  same  as  described  for 
the  standard  ACT. 

The  modified- ACTs  for  the  two  noise  levels  are  in  Fig.  6A,  and  the  synaptic- 
ACTs  are  in  Fig.  6B.  The  shapes  of  the  synaptic- ACTs  differ  from  the  synaptic  current 
because  from  one  sweep  to  another,  different  noise  schedules  were  given  that  produced 
variation  in  the  temporal  position  of  the  action  potential  with  respect  to  the  time  of 
occurrence  of  the  EPSC. 

FIGURE  6  NEAR  HERE 

The  most  obvious  differences  between  the  modified- ACTs  and  the  standard  ACT 
in  Fig.  5  are  a  difference  in  magnitude  and  a  change  in  shape.  The  late  positive  hump  in 
the  modified- ACTS  is  almost  a  thousand  times  smaller,  narrower  and  asymmetric 
compared  to  the  standard  ACT.  In  low  noise,  the  ratio  of  the  magnitudes  in  positive  to 
negative  components  in  the  trajectory  of  the  modified- ACT  are  nearly  equal.  In  high 
noise,  the  positive  phase  is  larger  and  more  nearly  symmetric  than  in  low  noise.  The 
phase  relation  between  the  synaptic-  and  modified-ACTs  changes  with  noise.  The 
zero-crossing  point  of  the  low-noise  modified- ACT  falls  near  the  peak  of  its 
corresponding  synaptic- ACT,  whereas  the  crossing  point  of  the  high-noise  modified- 
ACT  is  displaced  approximately  4  ms  earlier  with  the  peak  of  its  corresponding 
synaptic-ACT. 

As  noise  increases,  the  EPSC  plays  a  diminishing  role  in  driving  the  cell,  and  the 
modified- ACT  begins  to  approach  in  shape  and  magnitude  what  was  seen  previously 
■yyitE  the  standard  ACT.  There  is  also  a  progressive  loss  of  information  about  the 
synaptic  current  as  the  noise  increases.  This  is  a  reciprocal  relation:  The  information 
about  the  noise  and  the  synaptic  current  provided  by  observing  the  action  potential 
latencies  are  inversely  related.  As  the  noise  approaches  zero,  the  overall  ACT, 
consisting  of  the  combined  noise  and  synaptic  currents,  approaches  that  of  the  timing, 
shape  and  amplitude  of  the  EPSC. 


3.4.2  Using  the  modified-ACT  contingently:  observing  spike  trains  from  the  innt  frame  of 

reference 

The  standard  method  of  constructing  the  trajectories,  ignores  an  important 
source  of  information  about  the  circuit  in  which  the  neuron  participates  (We  use  the 
term  "circuit"  to  refer  to  the  temporal  structure  of  any  input  signal  that  might  occur 
naturally  or  be  applied  experimentally).  The  timing  of  the  action  potential  in  the  driven 
cell  depends  on  the  relation  in  the  sequence  of  the  driving  current-sources  with  respect 
to  each  other.  This  temporal  information  is  invisible  when  looking  only  at  the 
postsynaptic  neuron  and  ignoring  the  circuit  driving  its  activity.  The  histograms  of 
postsynaptic  spike  latency,  as  measured  with  respect  to  the  start  of  the  simulation, 
indicates  the  extent  of  this  information  (Fig.  6  C  and  D).  Noise  produces  a  slurred 
distribution  of  the  action-potential  latencies  over  a  wide  range,  and  the  distribution  is 
sensitive  to  the  level  of  the  noise.  As  the  noise  intensity  increases  there  is  a  shift  of  the 
mean  to  shorter  latencies,  a  broadening  of  the  distribution,  and  an  increased  frequency 
of  failures.  Each  bin  of  latencies  represents  the  effect  produced  by  particular  sets  of 
noise  signals. 

This  additional  information,  which  may  be  gained  by  using  the  input  circuit  as 
the  frame  of  reference,  causes  a  subtle  but  important  change  in  the  evaluation  of  the 
observations.  With  the  standard  ACT,  our  frame  of  reference  is  bound  to  the  action 
potential  in  the  postsynaptic  neuron.  But  using  the  contingency  of  the  latency 
histograms  our  observations  are  bound  to  the  time  frame  of  the  afferent  stream  because 
each  histogram  bin  represents  the  stimulus  conditions  that  generate  a  class  of  latencies. 
An  observer  can  exploit  this  binding  information  by  collecting  a  family  of  ACT's,  one 
for  the  set  of  current  impulses  associated  with  a  given  latency  bin  rather  than  a  single 
ACT  as  was  done  before.  Since  the  spike-evoking  EPSC  is  used,  these  ACTs  are 
modified  and  contingent.  To  distinguish  them  from  other  averaging  methods  above,  we 
refer  them  as  modified  contingent  ACTs  (modified-CACTs). 

In  Fig.  7  we  construct  the  histograms  so  as  to  divide  the  range  of  the  60,000 
latencies  in  the  low-  and  high-noise  experiments  into  regions  of  approximately  equal 
occupancy.  These  boundaries  are  used  as  criteria  for  collecting  multiple  ACTs  where  the 
inclusion  of  a  current  sweep  in  a  particular  class  is  contingent  on  the  class  of  latencies  it 
causes.  In  this  way,  there  is  one  pair  of  current  and  s)maptic  CACTs  corresponding  to 
each  of  the  five  regions  of  the  latency  histogram.  Since  the  averaging  process  is  bound 
to  the  stimulus  and  not  to  the  postsynaptic  action  potential,  it  is  possible  to  collect  a 
sixth  ACT  corresponding  to  the  class,  F,  of  current  samples  that  suppressed  action 
potentials.  This  trajectory  is  the  lowest  trace  in  Fig  7  (A  and  B)  that  has  the  most 
negative  current  dip  near  the  peak  of  the  EPSC. 
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Under  these  measurement  circumstances,  the  synaptic  CACT  becomes  a  constant 
equivalent  to  the  true  EPSC,  and  is  shown  by  the  single  synaptic  trajectory.  Variation  is 
restricted  to  the  modified  CACTs,  but  they  all  consist  of  scaled  versions  of  the  filter 
function  that  was  firsts  encountered  with  the  standard  ACT  shown  in  Fig.  4.  The 
magnitude  of  the  function  is  proportional  to  the  degree  to  which  the  noise  changes  the 
action  potential  latency.  The  peak  action  potential  advancement  is  caused  by  the  current 
trajectory  having  the  most  negative  early  phase  and  the  greatest  positive  late  phase. 
Conversely,  the  peak  latency  retardation  is  produce  by  the  trajectory  having  the  most 
positive  early  phase  and  the  most  negative  late  phase.  The  form  of  the  modified  CACTs 
does  not  change  with  noise  intensity,  and  neither  does  its  relationship  to  the  EPSC. 

Thus,  the  various  CACTs  shown  in  Fig.  7  accurately  reflect  the  distribution  of  the 
current  trajectories  that  were  presented  to  the  neuron  and  not  the  mechanism  by  which 
the  current  trajectories  were  chosen. 

FIGURE  7  NEAR  HERE 

It  is  apparent  from  this  that  the  sensitivity  of  the  ACTs  in  Fig.  6  to  the  amplitude 
of  the  noise  was  a  consequence  of  the  ignorance  of  the  observer  about  the  timing  of  the 
postsynaptic  action  potential  with  respect  to  the  activity  in  its  driving  network  and  does 
not  accurately  reflect  the  constancy  and  sensitivity  of  the  relation  between  the  two  input 
streams  (noise  pulses  and  EPSC)  and  the  postsynaptic  action  potential  latency.  This 
reflects  the  inability  of  an  ignorant  observer  to  extract  the  available  information  in  the 
ACT  rather  than  an  absence  of  information.  The  results  indicate  that  on  the  average 
knowledge  of  the  latency  of  the  action  potential  provides  information  about  the  shape 
of  the  noise  that  generated  it.  Conversely,  knowledge  of  the  average  shape  of  the 
stimulus  trajectory  contains  information  about  the  latency  of  the  action  potential. 

Importantly,  the  average  synaptic  current  is  not  significantly  affected  by  the 
noise  since  because  the  synaptic  event  and  the  averaging  process  are  both  bound  to  the 
stimulus  and  remain  constant.  From  Shannon  [Shannon,  1948  #726]  we  know  that 
because  the  s)maptic  current  is  constant  it  contributes  no  information  to  the  observation. 
Thus,  the  information  in  the  observed  variation  in  the  latencies  is  solely  related  to  the 
shape  of  the  noise  current  stream  near  the  synaptic  current. 

As  small  as  they  were,  the  modified- ACTs  in  Fig.  6  differed  from  zero,  but  the 
average  of  all  of  the  modified-CACTs  in  Fig.  7  should  be  zero  since  they  are  properly 
weighted  by  the  averaging  process.  To  resolve  this  apparent  paradox  we  need  to 
remember  that  the  construction  of  the  modified- ACTs  depended  on  the  action  potential 
as  a  reference  point,  and  this  required  that  the  F-class  trajectories  be  excluded.  They 
need  not  be  excluded  in  constructing  the  modified-CACTs  because  the  selection  method 
was  independent  of  the  action-potential  reference. 


The  significance  of  this  shift  in  the  frame  of  reference  of  the  observation  can  be 
appreciated  by  constructing  modified-CACTs  that  are  referenced  to  the  action  potential, 
much  as  is  done  in  constructing  the  standard  ACT.  This  shifts  the  frame  of  reference 
back  to  the  output  stream,  but  retains  the  contingency  aspect  by  grouping  the  latencies 
into  discrete  ranges  according  to  the  latency  of  the  action  potentials.  So,  now  the  times 
of  occurrence  of  the  action  potentials  are  fixed  by  definition  of  the  construction  method. 
The  uncertainty  that  was  previously  associated  with  the  time  of  occurrence  of  the 
postsynaptic  action  potential  is  transferred  to  the  interval  between  the  EPSC  and  the 
postsynaptic  action  potential. 

The  results  are  shown  in  Fig.  8.  The  five  histogram  bins  used  in  the  analysis  in 
Fig.  7  were  used  here  to  construct  modified-CACTs  from  the  reference  point  of  the 
action  potentials.  Since  action  potentials  are  used,  F-class  trajectories  could  not  be 
constructed.  Panels  A  and  C,  respectively,  show  the  effects  on  the  modified-CACTs  of 
low  and  high  noise.  Panels  B  and  D  show  the  synaptic-C ACTS  for  each  level  of  noise.  It 
can  be  seen  that  the  modified-CACTs  are  the  same  as  the  stimulus-bound  CACTs  in  Fig. 
7,  except  for  a  translation  related  to  the  different  spike  latencies  induced  by  them.  As  a 
result  of  this  translation  there  is  a  variable  causal  uncoupling  of  the  postsynaptic  action 
potential  from  the  afferent  current  stream.  The  duration  of  the  uncoupling  varies  with 
the  latency  class  of  the  modified-CACTs  and  attains  a  maximum  value  of  about  2.5  ms 
for  the  longest  latency  class  shown  here.  From  this  point  of  view  the  system  appears  to 
be  capable  of  faithfully  preserving  the  information  about  the  relation  of  noise  and 
synaptic  currents  over  variable  and  relatively  long  periods  of  time.  If  these  modified- 
CACTs  are  averaged  together  the  modified- ACTs  observed  before  for  the  two  noise 
levels  in  Fig.  6A  will  result  due  to  the  unequal  overlap  of  the  offset  population  of 
trajectories,  showing  that  the  information  in  the  modified-CACTs  is  contained  in  the 
modified- ACT.  It  is  a  matter  of  how  one  reads  the  data,  but  in  reading  it  one  way,  one 
losses  information  that  is  available  from  the  other. 

FIGURE  8  NEAR  HERE 

For  this  reason,  the  S5maptic-  CACTs  are  no  longer  constant,  indicating  the 
relative  ignorance  about  their  time  of  occurrence  with  respect  to  the  action  potential. 
Nonetheless,  the  relative  timing  between  the  synaptic  currents  and  the  noise  is  retained. 
For  reference,  the  curves  for  both  the  synaptic-  and  modified- ACTs  are  labeled 
numerically,  but  since  the  trajectories  all  progress  in  succession  numerically,  only  "1" 
and  "5"  are  shown.  Using  these  numbers,  it  can  be  seen  that  although  the  synaptic- 
ACTs  at  different  positions,  they  retain  the  same  phase  relation  to  their  corresponding 
modified-ACTs.  Thus,  as  in  the  case  of  the  modified- ACTS,  summing  the  synaptic- 
CACTs  in  Fig.  8  A  and  D  yields  the  corresponding  synaptic- ACTs  in  Fig.  6B  for  the 


two  noise  levels,  with  the  appropriate  time  between  the  modified- ACTs  and  the 
synaptic-ACTs. 

The  differences  in  the  various  ACTs  obtained  in  the  presence  of  synaptic  events 
depend  on  whether  the  observer  is  ignorant  of  presynaptic  action  potential  timing, 
postsynaptic  action  potential  timing  or  both.  It  appears  that  information  about  the 
driving  sources  stabilizes  and  maximizes  the  information  about  small  amplitude 
perturbations  that  can  be  extracted  by  observing  the  timing  of  the  action  potentials.  The 
answer  one  obtains  about  the  relationship  between  input  stimuli  and  the  action- 
potential  output  depends  in  important  ways  on  the  frame  of  reference  from  which 
questions  are  constructed  to  inquire  into  this  relationship. 

3.5  Time  Course  of  Single-Pulse  Perturbations  on  EPSC-Evoked  Spike  Latencies 

These  experiments  show  that  small  afferent  current  signals  significantly  affect 
action  potential  timing  over  long  periods  of  time  preceding  the  action  potential.  To  a 
first  approximation  the  relation  is  described  reasonably  well  by  linear  filter  models  even 
though  the  range  of  membrane  potentials  over  which  the  system  operates  under  these 
circumstances  considerably  exceeds  the  range  of  potentials  over  which  a  linear  model  is 
valid. 

In  order  to  understand  how  linear  relations  persist  over  non-linear  ranges  of 
membrane  potential  we  look  at  how  the  response  of  control  parameters  to  small,  sunple 
perturbing  currents  produce  changes  in  spike  timing  following  an  EPSC.  The  aim  wiU 
be  to  obtain  a  better  understanding  of  the  local  membrane  mechanisms  that  transform 
the  input  stream  into  the  output  stream  than  is  available  from  the  white-noise 
experiments  used  above. 

In  the  present  series  of  experiments,  the  same  EPSC  that  was  used  above  is 
presented  at  a  fixed  latency  in  each  simulated  sweep.  To  simplify  the  analysis,  the 
continuous  white  noise  is  replaced  by  a  delta  function,  approximated  by  a  single  current 
impulse  lasting  one  time  step,  is  presented  at  some  suitable  time  during  each 
independent  simulation  while  model  parameters  of  interest  are  monitored. 

Figure  9A  contains  four  superimposed  simulation  sweeps:  (1)  The  time  course  of 
the  voltage  response  produced  by  a  single  depolarizing  (solid  line)  or  (2)  a  single 
hyperpolarizing  (dashed  line)  current  impulse.  (3)  In  the  absence  of  an  EPSC,  the  thick 
solid  line  shows  the  voltage  time  course  in  response  to  the  current  impulse  alone, 
displaying  the  familiar  damped  oscillation  predicted  by  the  IRF.  (4)  The  voltage 
response  to  the  EPSC  when  no  conditioning  pulse  is  presented  (dotted  line).  At  this 
magnification,  all  three  EPSPs  are  shown  by  the  rapidly  rising  thin  soUd  line  that  begins 
at  11  ms.  The  nearly  vertical  dashed  line  just  before  the  20  ms  time  mark  shows  the 
descending  course  of  the  action  potential  that  was  accelerated  by  the  hyperpolarizing 


pulse.  The  most  important  feature  of  this  illustration  is  that  the  magnitude  of  the 
voltage  response  to  the  depolarizing  and  hyperpolarizing  pulses  is  less  than  0.2  mV  in 
both  cases  Despite  such  small  amplitudes,  it  will  be  shown  below  that  a  single  pulse  can 
affect  the  latency  of  EPSP-evoked  action  potentials  that  precede  the  onset  of  the  EPSP  by 
as  much  as  35  ms. 

EIGURE  9  HERE 

The  action  potential  produced  by  the  unperturbed  EPSC  is  shown  by  the  dotted 
trace  in  Fig.  9B.  Despite  the  weak  immediate  effects  produced  by  the  perturbing 
impulses,  the  latency  is  significantly  advanced  (dashed  trace)  by  the  presentation  of  the 
hyperpolarizing  pulse,  and  retarded  (solid  trace)  by  the  depolarizing  pulse.  The 
retardation  of  the  action  potential  caused  by  the  depolarizing  impulse  is  greater  than 
the  advance  caused  by  hyperpolarizing  impulse  of  the  same  magnitude.  The  voltage 
response  to  the  impulses  preceding  the  EPSC  are  invisible  at  the  display  scale  used  in 
Fig.  9B. 

Except  for  the  perturbing  current  impulses,  which  are  an  arbitrary  imposed 
signal,  everything  else  in  these  simulations  is  generated  by  the  dynamics  of  the  model. 
The  changes  in  spike  latency  must,  of  course,  work  through  the  sodium  and  potassium 
gate  parameters  in  the  simple  membrane  used  here.  The  evolution  of  the  n  and  m  gate 
parameters  underlying  the  advancement  of  the  action  potential  induced  by  the 
hyperpolarizing  pulse  are  shown  in  Fig.  9  D  and  E,  respectively.  The  response  of  the 
membrane  potential  to  the  hyperpolarizing  pulse  shown  in  Fig.  9 A  is  shown  again  in 
Fig.  9C  for  reference.  Gate  m  inactivates  rapidly  but  has  largely  recovered  by  the  time 
of  the  EPSC  onset  while  the  slower  potassium  gate  n  is  still  near  its  nadir.  This  residual 
inactivation  which  advances  the  onset  of  the  action  potential. 

A  full  experiment,  in  which  current  impulses  of  various  magnitudes  were 
presented  at  0.25  ms  intervals  over  a  30  ms  time  span  before  the  EPSC  and  partially 
overlapping  it,  is  shown  in  Fig.  10.  The  time  courses  of  the  synaptic  conductance  and 
the  response  of  the  membrane  potential  to  it  are  shown  for  reference  in  Fig.  10  A  and  B, 
respectively. 

FIGURE  10  HERE 

The  solid  and  dotted  lines  that  appear  in  Fig.  IOC  and  D,  above  the  membrane- 
potential  recording,  show  the  change  in  action  potential  latency  as  a  function  of  impulse 
presentation  time  for  different  current  magnitudes.  The  lines  are  isopleths:  i.e.,  each 
line  connects  all  of  the  points  obtained  from  impulses  of  the  same  magnitude  presented 
at  the  times  indicated.  The  dotted  latency-change  curves  were  obtained  using 


hyperpolarizing  pulses,  and  the  solid  curves  were  obtained  using  depolarizing  pulses. 
Some  of  the  dotted  curves  are  open.  This  indicates  that  the  postsynaptic  action 
potential  was  suppressed  when  the  impulses  were  presented  over  that  range  of  times. 
Similar  sets  of  curves  are  shown  for  the  membrane  under  normal  polarization  (Fig.  lOD) 
and  under  sustained  hyperpolarization  (Fig.  IOC).  The  curves  for  sustained 
depolarization  are  not  shown. 

These  curves  strongly  resemble  the  filter  functions  seen  earlier  with  the  IRF  (Fig. 
2),  ACT  (Fig.  4)  and  CACTs  (Figs.  7  and  8),  and  the  different  phases  of  the  curves  in  Fig. 
!0  are  numbered  to  coincide  with  the  numbering  scheme  for  the  IRF  and  ACT.  Since 
currents  are  used  in  these  filter  functions  and  latencies  are  used  in  Fig.  10,  the 
deflections  of  the  i/o  curves  in  Fig.  10  D  have  the  opposite  deflection  relative  to  those 
seen  earlier.  For  example,  the  solid-line  curves  in  Fig.  10  A,  correspond  to  the 
trajectories  in  Fig.  7  that  have  positive  current  deflections  just  after  the  peak  of  the 
EPSC.  Similarly,  the  solid-line  curves  in  Fig.  10  D  have  inverse  deflections  by 
comparison  to  the  ACT  shown  in  Fig.  4. 

Overall,  there  are  two  major  phases  of  opposite  effect.  The  larger  phase  peaks 
about  4  ms  before  the  normal  action  potential  and  the  earlier,  smaller  and  inverted  peak 
at  19  ms,  12  ms  before  the  action  potential.  Times  are  approximate  since  as  the  spike 
latency  changes  the  time  from  any  feature  on  the  curve  to  the  start  of  the  action 
potential  will  change  with  it.  This  is  the  same  distinction  that  was  made  earlier  with  the 
stimulus-bound  and  action  potential-bound  CACTs.  This  illustration  adopts  the 
afferent-stream  frame  of  reference.  As  seen  before  in  the  CACTs  the  maximum 
perturbing  influence  occurs  at  26  ms,  a  little  less  than  1  ms  after  the  peak  EPSC.  There  is 
a  null  point  at  22.44  ms  where  no  current  has  any  detectable  perturbing  influence  on  the 
action  potential  latency  which  corresponds  to  the  crossover  points  of  the  stimulus- 
bound  CACTs.  Also  like  the  CACTs,  perturbing  influences  which  tend  to  retard  the 
occurrence  of  the  action  potential  are  stronger  than  those  which  advance  the  action 
potential.  A  given  depolarizing  current  impulse  at  26  ms  advances  the  action  potential 
less  than  a  hyperpolarizing  impulse  of  the  same  magnitude  will  retard  it.  The  disparity 
is  seen  most  easily  with  the  ability  of  the  hyperpolarizing  impulses  to  retard  the  action 
potential  by  several  ms  while  depolarizing  impulses  of  the  same  magnitude  are  imable 
to  advance  the  action  potential  more  than  1  ms  over  the  range  of  magnitudes  reported 
here. 

A  constant  hyperpolarizing  current  changes  the  i/ o  space  as  expected  from  the 
IRF.  The  null  point  is  moved  to  an  earlier  time  and  the  early  phase  is  prolonged  and 
reduced  in  amplitude  compared  to  the  late  phase.  Notice  that  the  decay  of  the  late 
phase  (26-31  ms)  does  not  change  during  hyperpolarization.  The  broadening  of  the  late 
phase  peak  is  accomplished  by  a  broadening  of  its  rising  phase  which  moves  the  null 
point  to  an  earlier  time  in  the  sweep. 


A  third,  earlier,  phase  in  the  isopleths  of  Fig.  lOD ,  noted  by  "3",  is  visible  under 
expansion  of  the  vertical  axis,  but  becomes  increasingly  larger  as  the  membrane  is 
depolarized  (not  shown).  Thus,  the  span  of  preconditioning  history  with  detectable 
influences  on  the  timing  of  the  action  potential  can  easily  reach  30-35  ms  into  the  past 
before  the  onset  of  the  EPSC,  depending  on  the  polarization  of  the  membrane. 

The  experiments  shown  in  Figs.  2, 4, 6-8  and  10  have  addressed  a  wide  range  of 
response  characteristics  of  the  membrane.  The  IRF,  as  used  to  generate  the  results 
shown  in  Fig.  2,  involved  sufficiently  weak  stimulation  that  the  membrane  remained 
within  its  linear  response  range.  The  random  current  used  to  generate  the  ACT  (Fig.  4) 
forced  the  membrane  into  its  non-linear  range.  The  EPSC  used  to  obtain  the  modified- 
ACTs  (Fig.  6)  and  the  modified-CACTs  (Figs.  7,8)  also  operated  over  the  non-linear 
range  while  the  noise  which  perturbed  it  was  within  the  linear  range.  Similarly,  the 
isopleths  in  Fig.  10  were  obtained  with  current  impulses  which  by  themselves  would 
not  exceed  the  linear  range  of  the  membrane.  The  EPSC  acts  as  an  amplifier  such  that 
the  small  perturbations  cause  significant  changes  in  the  latency  of  action  potentials. 
Under  all  of  these  conditions,  the  history  of  conditioning  effects  appear  to  have  similar 
temporal  structure. 

3.6  Perturbation  and  time-evolution  of  gates  m  ,h ,  and  n 

The  structure  in  the  history  of  these  conditioning  effects  must,  as  noted  above,  be 
related  to  the  time  and  voltage  dependencies  of  the  ion  gates  in  the  Hodgkin-Huxley 
model.  Figure  11  B-D  shows  the  m  and  n  gate  dynamics  that  produce  the  multiphased 
i/ o  curves.  Figure  11 A  shows  two  isopleths,  one  obtained  using  +10  x  10"^  Amp  pulses, 
that  retard  the  latencies  over  the  first  part  of  the  curve  and  then  advance  them,  and  one 
using  -10  X  10"^  Amp  pulses  that  initially  advance  the  spike  latencies  and  then  retard 
them.  There  are  five  labeled  points  of  interest  placed  on  the  curves  to  mark  important 
locations.  Point  a  represents  the  normal  or  unperturbed  condition.  Point  b  is  placed  at 
19  ms,  the  presentation  time  at  which  the  hyperpolarizing  impulse  produces  its 
maximum  advancement  of  the  action  potential.  Point  c  is  at  22.44  ms,  the  null  point, 
where  current  impulses  produce  no  change  in  action  potential  latency.  Point  d  is 
placed  on  the  baseline  at  26  ms.  It  belongs  on  the  -10x10"^  isopleth  but  at  that 
presentation  time  the  action  potential  is  suppressed  and  that  region  of  the  isopleth  does 
not  exist.  Point  e  is  near  the  terminal  point  of  no  effect  where  the  action  potential  is  no 
longer  susceptible  to  perturbing  influences.  It  is  shown  at  30  ms  but  is  really  somewhat 
later  just  before  the  moment  where  the  membrane  potential  normally  crosses  the 
criterion  detection  threshold  (-30  mV).  In  the  absence  of  perturbing  influences  the 
membrane  potential  crosses  that  threshold  at  30.96  ms  (horizontal  dashed  line).  The 
EPSC  begins  at  25  ms.  In  the  following  discussion,  we  examine  the  results  of  the 


experiments  using  -10  x  10'^  Amp  impulses  to  examine  how  the  ion  gates  are  related  to 
the  isopleths. 

FIGURE  11  HERE 

The  other  three  panels  in  Fig.  11  show  the  evolution  of  the  potassium  activation 
gate  parameter  n  and  the  sodium  activation  gate  parameter  m  during  three 
representative  experiments.  We  first  discuss  the  general  features  of  these  diagrams,  and 
then  go  in  to  the  details  of  each  feature. 

Points  relating  to  the  labeled  points  in  Fig.  11 A  are  placed  on  the  trajectories.  In 
all  panels,  the  dotted  line  is  the  trajectory  from  the  simulation  with  the  EPSC  alone;  it  is 
the  unperturbed  or  control  trajectory.  The  other  trajectories  are  the  result  of  paired 
pulse /EPSC  stimulations.  The  dashed  line  is  the  trajectory  resulting  from  the  simulation 
in  which  the  impulse  was  presented  the  at  26  ms.  This  trajectory  is  launched  from  the 
same  point  as  the  control  trajectory,  so  the  high-resolution  graphs  of  the  laimch  points 
show  a  combined  trajectory  made  up  of  dots  and  dashes.  The  solid  line  is  the  trajectory 
from  the  simulation  where  the  impulse  was  presented  at  19  ms.  Thus,  the  illustrations 
exhibit  one  normal  (or  control)  and  two  extreme  trajectories,  the  maximum  retardation 
(dashed  line)  and  the  maximum  advancement  (solid  line)  of  the  action  potential.  The 
trajectory  that  results  from  the  presentation  of  the  perturbing  impulse  alone  is  shown  as 
solid-line  arc.  In  all  cases,  the  sodium  gate  activation  parameter  m  is  presented  on  the 
vertical  axis  and  the  potassium  gate  activation  parameter,  n,  is  shown  on  the  horizontal 

axis. 

The  significance  of  these  graphs  is  that  they  constitute  a  reduced  phase  space 
representation  of  the  underlying  dynamics.  By  showing  the  relation  between  dynamic 
variables  as  the  system  evolves,  the  phase  space  images  are  especially  useful  for 
understanding  how  the  variables  interact  to  produce  the  observed  behavior.  The  m-n 
plane  is  a  reduced  phase  space  because  it  is  incomplete.  A  plot  of  the  complete  four 
dimensional  phase  space  (V,tn,h,n)  is  not  feasible  and  an  incomplete  or  reduced 
representation  of  the  phase  space  is  needed.  The  variables  are  not  entirely  independent 
of  one  another  [Kepler,  1991  #958;  Fitzhugh,  1961  #926].  Because  V  and  m  behave 
similarly,  and  because  h  and  n  behave  similarly,  we  use  a  reduced  phase-plane 
constructed  from  one  variable  from  each  pair  for  clarity.  This  is  sufficient  for  exploring 
the  dynamics.  It  is  incomplete,  but  it  is  adequate  as  a  tool  for  assisting  thinking  about 
the  underlying  process  where  subthreshold  perturbations  affect  the  generation  of  action 
potentials. 

Figure  IIB  shows  a  medium  close-up  of  the  m-n  phase  plane.  Time  is  an  implicit 
parameter  and  the  general  flow  is  from  the  bottom  left  to  the  upper  right  as  the  sodium 
and  potassium  activation  parameters  increase  during  depolarization.  The  curves 


continue  their  paths  off  the  graph  upward  and  to  the  right  as  the  action  potential 
progresses,  leading  eventually  in  a  clockwise  fashion  back  to  the  original  resting  state  at 
point  a  after  the  action  potential. 

Trajectories  to  the  left  of  the  control  (dotted)  trajectory  are  advanced  (reduced 
latency)  and  the  trajectories  to  its  right  are  retarded  (increased  latencies).  It  can  be  seen 
that  the  19  ms  perturbation  moves  the  trajectory  (solid  line)  to  the  left  of  the  control 
trajectory.  The  point  d  marks  the  state  of  the  gates  where  the  perturbation  has  its 
maximum  retardation  effect  (26  ms),  and,  in  this  case,  the  action  potential  is  aborted 
(dashed  line).  To  give  a  reference,  the  point  on  the  unperturbed  trajectory  at  which  the 
voltage  crosses  -30  mV  is  labeled  e.  This  occurs  at  30.96  ms,  the  unperturbed  action 
potential  latency;  refer  to  the  Fig.  11 A  for  temporal  orientation.  Point  d  occurs  1  ms 
after  the  beginning  of  the  EPSP  (25  ms)  at  point  a  and  is  about  5  ms  before  point  e. 

A  higher  resolution  graph  of  the  initial  rising  phase  of  the  trajectories  is  shown  in 
Fig.  lie.  The  divergence  of  the  26  ms  trajectory  beginning  at  point  d  is  apparent.  The 
hyperpolarizing  current  impulse  drives  the  voltage  down.  This  inactivates  m,  forcing 
the  trajectory  downward  and  also  inactivates  n,  forcing  the  trajectory  to  the  left.  The 
reduced  activation  of  the  sodium  conductances  peaks  near  the  closest  approach  to  the 
separatrix  which  lies  above  the  trajectories  in  this  panel.  The  response  time  of  m  is 
short  and  the  decrease  in  sodium  activation  dominates  this  response,  preventing  the 
trajectory  from  rising  above  the  separatrix.  (The  location  of  the  separatrix  is  difficult  to 
define,  and  will  be  examined  in  a  following  section). 

The  interrelationships  of  all  of  the  trajectories  may  be  imderstood  using  Fig.  IID 
which  is  an  extreme  close-up  of  the  foot  of  the  trajectories.  The  arc  acb  is  the 
subthreshold  trajectory  of  the  response  of  m  and  n  to  the  perturbation  produced  by  the 
hyperpolarizing  impulse  (-10  x  10'^  Amp).  The  arc  begins  at  point  a  and  swings 
clockwise.  In  the  absence  of  an  EPSC,  the  hyperpolarizing  pulse  would  cause  a 
succession  of  m-n  states  that  follow  the  solid-line  curve  that  starts  at  point  a  and 
continues  through  points  c,  b,  and  back  to  a  eventually.  The  arc  is  broken  between  b 
and  a  in  order  to  show  the  launch  positions  of  the  control  and  the  two  extreme-case 
trajectories  that  will  be  used  to  visualize  how  the  perturbations  affect  spike  genesis.  The 
control  and  the  maximally  delayed  trajectories  are  shown  rising  as  a  combined  dashed 
and  dotted  trajectory  from  point  a.  The  maximally  advanced  trajectory  is  shown  by  the 
solid  line  emerging  from  point  b.  The  loop  is  a  result  of  the  fact  that  although  the 
hyperpolarization  of  the  membrane  induces  a  decrease  in  both  m  and  n,  m  responds 
more  quickly  than  n  (as  shown  in  Fig.  9  C-E)  causing  a  nearly  vertical,  downward 
initial  motion.  Later,  as  n  responds  more  slowly,  it  pulls  the  curve  out  to  the  left. 
Recovery  of  m  is  well  under  way  while  n  is  still  decreasing  when  the  EPSC  arrives.  (A 
depolarizing  pulse  of  equal  magnitude  would  produce  a  mirror  image  of  the  aeba  loop 
in  a  clockwise  flow,  initially  up  and  to  the  right). 


Point  a  is  the  sole  fixed  point  in  the  complete  phase  space  and  depicts  the 
normal  resting  position  of  the  membrane,  unperturbed  by  current  pulses  or  synaptic 
currents.  It  is  from  this  point  that  all  trajectories  begin.  If  the  impulse  is  presented  before 
the  arrival  of  the  EPSC,  then  the  impulse  evokes  the  solid-line  loop  described  above. 
The  earlier  the  impulse  is  presented  before,  the  more  fully  the  loop  will  evolve  before  it 
is  interrupted  by  the  EPSC.  If  the  impulse  is  presented  after  the  arrival  of  the  EPSC,  then 
the  perturbed  and  unperturbed  trajectories  coincide,  as  shown  by  the  dashed-line 
trajectory,  until  the  impulse  is  presented.  The  rapidly  rising  trajectories  are  the  initial 
response  of  m  and  n  to  the  synaptic  current. 

When  the  hyperpolarizing  perturbation  impulse  is  presented  at  19  msec,  6  ms 
before  the  onset  of  the  EPSC,  the  initial  decrease  in  sodium  activation  has  almost 
completely  recovered,  but  n  is  still  significantly  reduced.  When  no  EPSC  is  presented, 
the  combined  time  courses  of  tn  and  n  that  occur  in  response  to  the  unpulse  produce  a 
phase-plane  arc  upward  and  to  the  right  of  point  c  .  Thus,  the  trajectory  launched  from 
point  b  begins  to  the  left  of  the  control  trajectory  because  of  the  residual  suppression  of 
the  potassium  activation  and  almost  normal  sodium  activation,  and  it  is  this  persistence 
of  the  potassium  inactivation  that  reduces  the  action  potential  latency. 

Point  c  denotes  the  location  on  the  acb  loop  where  the  EPSC  would  interrupt  the 
normal  evolution  of  the  response  of  the  gates  to  the  hyperpolarizing  perturbation  pulse 
if  it  had  been  presented  at  22.44  ms.  The  EPSC  arrives  2.56  ms  later.  At  that  point  there 
is  residual  inactivation  of  both  tn  and  n.  The  balance  of  the  two  inactivations  is  such 
that  the  action-potential  advancing  influence  of  the  decreased  potassium  channel 
activation  is  exactly  offset  by  the  action  potential  retarding  effect  of  the  decrease  in  m 
so  that  there  is  no  appreciable  change  in  action  potential  latency.  Although  it  is  not 
included  here,  a  trajectory  emanating  from  point  c  in  Fig.  IID,  would  begin  to  the  left 
of  the  19  ms  perturbation  trajectory  emanating  from  point  b  and  then  cross  over  it  in 
order  to  approach  the  normal  (dotted)  trajectory  and  yield  the  same  action  potential 
latency  as  the  control  trajectory  emanating  from  point  a  .  As  required  by  the  Jordan 
(;urve  theorem  and  the  theorem  of  Poincare-Bendixson  (see  discussion  in  (Hofbauer  and 
Sigmund,  1988),  p.  149,  phase-plane  trajectories  of  deterministic  functions  cannot 
intersect.  The  fact  that  they  appear  to  cross  here  is  due  to  the  incompleteness  of  the 
reduced  two-dimensional  phase  plane  employed  in  constructing  Fig.  11  B-D. 

Many  trajectories  could  be  shown  emanating  from  the  baseline  loop  along  acb  in 
Fig.  IID;  ones  are  shown  only  at  points  a  and  b  in  order  to  allow  visualization  of  the 
loop  itself.  This  shows  how  small,  subthreshold  perturbations  impose  distinct  initial 
conditions  from  which  similar,  closely  packed  responses  arise.  The  smallness  of  the 
these  initial  conditions  can  be  appreciated  by  comparing  the  dimensions  of  the  baseline 
arc  to  its  phase  space  locations  in  Fig.  IIB.  Since  the  perturbation  pulses  used  to  obtain 
the  isopleths  had  a  presentation  range  of  25  ms  before  the  EPSC,  most  of  the  acb  loop  in 


Fig.  IID  consists  of  m  -n  values  obtained  within  0.25  mV  of  the  resting  potential.  All 
take-off  points  of  the  trajectories  on  the  baseline  loop  are  within  the  linear  response 
range  of  the  membrane. 

Once  the  EPSC  begins,  the  voltage  of  the  membrane  shifts  rapidly  into  nonlinear 
regions.  An  indication  of  this  effect  is  shown  in  Fig.  12.  In  all  panels,  the  solid-line 
shows  the  trajectory  leading  to  maximal  action-potential  advancement  (perturbation 
pulse  given  at  19  ms),  and  the  dashed  lines  represents  the  trajectory  leading  to  maximal 
retardation  (pulse  given  at  26  ms),  as  described  above.  The  remainder  of  the  baseline 
loop  is  made  up  of  unfilled  circles,  each  of  which  indicates  the  starting  point  of  a 
trajectory  for  a  different  impulse  presentation  time.  Above  the  baseline  loop  are  four 
similar  loops  where  each  new  loop  represents  a  slice  through  the  whole  family  of 
trajectories  at  different  times  during  the  simulation.  Each  loop  is  separated  from  the 
previous  loop  by  0.12  ms.  For  clarity,  only  the  extreme  trajectories  and  the  null  point 
trajectory  are  connected  by  lines.  In  Fig.  12  the  scales  in  all  four  panels  are  the  same. 
Compare  these  with  scales  in  Fig.  IIC  for  orientation. 

FIGURE  12  here 

Panel  B  shows  two  loops  taken  shortly  after  the  divergence  of  the  maximal  delay 
trajectory  at  point  d  of  Fig.  11.  The  course  of  the  unperturbed  trajectory,  and  of  the 
trajectories  that  will  be  produced  by  impulse  perturbations  other  than  the  one  causing 
maximal  delay,  represented  by  the  two  lone  points  lying  at  the  right  end  each  loop, 
approximately  half  way  between  the  solid  and  dashed  trajectories.  As  the  maximal- 
delay  trajectory  continues  to  diverge  from  the  unperturbed  trajectory,  other  non- 
maximally  delayed  trajectories  emerge  successively  from  behind  the  circle  residing  on 
the  dashed  line.  These  are  bom  as  impulses  are  presented  at  earlier  times  in  the 
simulation  with  respect  to  point  d.  A  total  of  six  such  trajectories  have  emerged  from 
the  first  loop  in  Fig.  12C,  and  seven  in  the  second  loop.  Also  shown  in  Fig.  12C  is  that 
the  null  (dotted-line)  trajectory  that  originated  from  point  c  in  Fig.  IID  has  nearly 
converged  with  the  circle  in  each  loop  representing  the  unperturbed  trajectory.  Along 
the  way,  the  null  trajectory  has  crossed  over  the  maximally  advanced  trajectory. 

The  convergence  between  the  point-c  and  point-a  trajectories,  however,  is  only  a 
false  impression  of  the  two  dimensional  projection  of  the  illustration.  This  impression 
also  indicates  that  the  points  on  the  c  -trajectory  are  advanced  with  the  points  on  the  a- 
trajectory.  In  higher  dimensions,  however,  it  could  be  seen  that  the  two  points  lie  on 
opposite  sides  of  the  cross  section  of  a  tube.  These  two  m  -n  states  are  sufficiently  close 
to  one  another  that  they  produce  similar  action  potential  latencies. 

Thus,  in  Fig.  12A-D,  the  m  -n  reduced  phase  space  represents  a  2-dimensional 
projection  of  an  expanding  cone  of  states.  Each  time-slice  or  loop  through  the  cone 


appears  as  an  ellipse  The  effect  of  the  EPSC  is  to  rotate  and  expand  the  initial  loop. 
Dividing  the  ellipses  roughly  in  half  horizontally  in  Fig.  12D,  the  top  half  contains 
trajectories  that  lead  to  action  potentials,  where  the  shortest  latency  is  represented  by 
the  top  solid  line.  The  bottom  half  of  the  ellipses  contain  trajectories  for  which  action 
potentials  are  not  generated,  where  the  maximal  suppression  of  the  m  -n  states  is  shown 
by  the  dashed  trajectory. 

It  is  also  possible  to  relate  these  time-slices  to  the  structure  of  the  isopleths  in  Fig. 
IIA.  The  radial  position  of  the  points  in  the  time  slices  represent  points  along  the 
isopleth  produced  by  the  hyperpolarizing  pulse.  Starting  with  the  solid-line  trajectory 
of  the  second  time  slice  shown  in  Fig.  12C  and  counting  clockwise,  the  first  2  trajectories 
represent  pulse  presentation  times  between  points  a  and  b  on  the  isopleth  (0-19  ms). 
Counting  counterclockwise,  the  first  8  trajectories  represent  pulse  presentation  times 
over  the  isopleth  from  point  b  to  point  c  and  part  of  the  way  up  the  segment  between 
c  and  d  (19-26  ms).  Both  of  these  sets  of  trajectories  lead  to  action  potentials.  Starting 
at  the  dashed-line  trajectory  and  counting  clockwise,  the  first  six  trajectories  arise  from 
impulses  presented  between  points  c  and  d .  In  the  counterclockwise  direction,  the  first 
5  trajectories  emerge  from  pulses  presented  between  points  d  and  e  .  Neither  of  these 
sets  generate  action  potentials.  The  effects  along  the  various  portions  of  the  time  slices 
are  graded,  such  that  there  is  either  a  progressive  change  in  the  latency  of  the  action 
potentials  associated  with  the  m-n  trajectories,  or  progressive  changes  in  the 
depolarization  of  the  membrane  potential  for  trajectories  that  do  not  lead  to  action 
potentials. 

Four  conclusions  may  be  summarized  from  the  above  findings  on  the  reduced 
phase  space. 

(1)  The  sample  series  of  slices  in  Fig.  12  appear  to  be  transections  of  the  surface  of 
a  tube  or  cone  that  expands  in  time. 

(2)  By  labeling  and  examining  the  positions  of  individual  circles  in  each  time 
slice,  it  can  be  seen  that  the  stretching  of  the  surface  of  the  cone  is  due  to  the  relative 
motion  of  the  trajectories  such  that  they  rotate  and  separate  from  one  another  but  do  not 
change  their  relative  position.  Above  the  separatrix,  the  trajectories  comprise  an 
expanding  hemisection  of  a  tube  whose  cross  section  resembles  an  inverted  U  that  has 
the  19  ms  trajectory  at  its  apex.  Expansion  continues  until  the  action  potential  is 
generated.  On  the  rising  phase  of  the  action  potential  the  trajectories  remain  more  or 
less  parallel  to  one  another.  Below  the  two  separatrices,  the  trajectories  form  an 
expanding  U  whose  base  has  the  26  ms  trajectory.  Expansion  continues  until  the 
postsynaptic  response  plateaus.  Thereafter,  there  is  contraction  of  state  space  until  the 
membrane  returns  to  its  resting  value. 

(3)  The  stretching  of  the  surface  of  the  cone  pulls  the  dotted-line  trajectory,  which 
emerges  from  point  c  on  the  baseline  slice,  away  from  the  adjacent  solid-line  trajectory. 


which  emerges  from  point  b  .  This  relative  motion  of  the  two  trajectories  on  the  surface 
of  the  cone  produces  the  visual  impression  that  the  two  cross.  It  also  shows  that  the 
apparent  change  in  relative  positions  of  points  on  the  surface  of  the  structure  is  due  to  a 
rotation  of  the  structure  and  is  not  due  to  a  change  in  relative  radial  positions  of  points 
on  the  surface.  The  trajectories  remain  separate  and  distinct,  they  don't  cross  or  mix. 

(4)  Most  important  is  that  the  effect  of  the  nonlinearities  engendered  by  the  EPSC 
is  simply  to  stretch  the  structure  of  the  baseline  phase-space  loop  produced  by  the 
perturbation  impulses. 

3.7  Relation  of  spike  thresholds  to  the  phase-space  separatrix 

The  separatrix  divides  all  trajectories  into  those  which  lead  to  an  action  potential 
and  those  which  do  not.  The  two  sets  of  trajectories  represent  alternative  basins  of 
attraction  to  the  fixed  point.  Although  the  squid  axon  described  by  Hodgkin  and 
Huxley's  equations  has  no  true  threshold,  the  separatrix  may  be  viewed  as  nonsingular- 
point  threshold,  which  Fitzhugh  [Fitzhugh,  1955  #929;  Fitzhugh,  1960  #927;  Fitzhugh, 
1961  #926]  has  termed  QTP  (quasi-threshold  phenomena). 

The  separatrix,  the  set  of  trajectories  which  belong  to  no  basin  of  attraction, 
serves  the  purpose  of  the  threshold.  But,  because  the  separatrix  belongs  to  no  basin  of 
attraction  any  trajectory  on  the  separatrix  will  not  approach  the  fixed  point  in  finite 
time.  Therefore,  as  a  trajectory  approaches  the  separatrix  from  above  the  latency  of  the 
action  potential  will  approach  infinity.  As  a  trajectory  approaches  the  separatrix  from 
below,  the  duration  of  the  repolarization  will  approach  infinity.  This  effect  is  what  gives 
importance  to  the  separatrix  as  an  organizing  structure  in  phase  space. 

Because  they  belong  to  no  basin  of  attraction  separatrices  have  probability  zero 
and  can  be  neither  experimentally  discovered  nor  analytically  determined.  As  a 
compromise  we  estimate  the  separatrix  by  bracketing  it  and  assuming  the  true 
separatrix  lies  somewhere  between  them.  We  used  a  current  impulse  to  drive  the  cell 
(no  FPSC)  for  this  estimation  by  adjusting  its  amplitude  to  thirteen  significant  figures  by 
trial  and  error  until  the  sub-  and  supra-threshold  responses  were  virtually  identical  for 
over  12  msec.  The  suprathreshold  stimulus  amplitude  for  the  cell  dimensions  used  here 
was  found  to  be  5.123152366087  x  10"H  Amp  and  the  subthreshold  value  was 
5.123152366086  x  lO'^l  Amp;  the  membrane  voltage  changes  produced  by  these  two 
stimuli  are  shown  by  the  traces  in  Figure  13A.  The  m-n  plane  relating  to  both  of  these 
voltage  traces  is  shown  in  Fig.  13B,  and  in  higher  magnification  in  Fig.  13C.  In  Fig.  13D, 
the  initial  portion  of  the  m-n  separatrix  phase  space  trajectory  is  shown  overlapped 
with  the  trajectories  originating  from  points  a,  b,  c  and  d  in  Fig.  IIB  using  a  .001  ms 
integration  step.  Keep  in  mind  that  this  image  of  the  separatrix,  like  the  trajectories 
projected  onto  the  reduced  phase  space,  is  incomplete.  The  separatrix  is  a  four 


dimensional  hyper-line  in  the  space  of  (V,m,h,n),  not  a  simple  line.  Trajectories 
originating  from  these  points  are  initially  lower  than  the  separatrix  because  of  the 
temporal  separation  between  the  time  at  which  the  pulse  was  given  and  the  onset  of  the 
EPSC.  However,  as  these  trajectories  cross  the  separatrix,  it  can  be  seen  that  the 
separatrix  takes  on  a  position  roughly  near  the  midpoint  between  the  trajectories  for  the 
most  advanced  action  potential  latency  (top  solid  line)  and  the  most  retarded  latency 
(bottom  dashed  line).  This  position  is  equivalent  to  the  lateral  mid-point  position  on  the 
time  slice  loops  shown  in  Fig.  12C.  As  clearly  illustrated  by  the  latency  shown  in  Fig. 
13D, 

FIGURE  13  HERE 
5  Discussion 

Our  principle  interest  here  is  to  understand  the  nature  of  information  processing 
at  the  level  of  single  neurons  in  terms  of  an  interpretation  of  known  biophysical 
mechanisms.  For  information  to  be  processed,  it  has  traditionally  seemed  reasonable  to 
accept  the  notion  that  there  must  be  some  meaningful  relation,  supported  by  known 
mechanisms,  between  the  information  entering  a  neuron  and  the  information  leaving  a 
neuron  although  it  need  not  necessarily  be  simple  or  obvious.  We  call  such 
transformations  "single-neuron  computation"  and  use  it  as  the  foundation  concept  for 
the  inquiries  presented  here  and  in  subsequent  papers.  We  shall  discuss  our  findings  as 
they  relate  to  the  ability  of  linear  membrane  impedance  responses  to  subserve 
input /output  transformations,  and  to  the  extent  to  which  these  transformations  are 
reliable. 

The  input/output  transformations  balance  a  difficult  design  problem  in  which 
neurons  must  satisfy  multiple  and  sometimes  contradictory  constraints.  On  the  one 
hand  the  neuron  must  accurately  propagate  information  that  it  receives.  This  requires  a 
predictable  relation  between  the  input  and  the  output  of  a  neuron  so  that  an  observer  of 
the  output  can  obtain  reliable  knowledge  about  the  input.  Balanced  against  this  need 
for  predictability  is  the  need  to  produce  novel  outputs  as  the  neuron  combines 
information  from  diverse  sources  (such  as  contemporary  sensory  images,  accumulated 
experience,  or  changes  in  the  level  of  neurochemical  modulation  acting  on  the 
membrane)  to  produce  an  output  appropriate  to  the  circumstances  that  may  differ  in 
significant  and  unexpected  ways  from  the  form  of  the  input. 

We  have  explored  how  membrane  properties  may  achieve  aspects  of  these 
design  constraints  by  looking  at  the  dependence  of  spikes  on  current  injection  in  models 
of  simple  but  plausible  neurons  where  the  input  signal  varies  in  size  and  form  from 
small  and  random  to  large  and  non-random. 


A  generic  computer  model  preparation  was  used  to  allow  rigorous  control  over 
input  and  to  be  able  to  observe  otherwise  unobtainable  internal  state  parameters.  One 
of  our  main  observations  is  that  surprisingly  small-amplitude  signals  make  important 
contributions  to  firing  patterns.  Measuring  and  controlling  small  input  signals  is 
expensive  in  terms  of  effort  and  ingenuity  in  real  life.  It  is  therefore  desirable  to 
understand  how  and  why  small  signals  might  be  important  to  neural  function  by 
looking  at  them  first  under  ideal  conditions  that  are  only  possible  in  a  model. 

Toward  this  end,  we  have  modeled  a  single  compartment  neuron  with  the 
membrane  electrical  properties  of  squid  giant  axon.  This  model  does  not  and  is  not 
intended  to  realistically  describe  any  particular  neuron.  No  neuron  has  exactly  this 
impedance.  But  all  neurons  do  have  an  impedance,  and  it  is  the  need  for  a  general 
understanding  of  how  impedance  relates  the  input  of  a  neuron  to  its  firing  pattern  that 
we  address  here.  The  properties  of  the  squid  axon  were  chosen  in  particular  because 
they  have  been  the  object  of  extensive  research  and  are  simple  enough  to  understand 
while  still  being  complex  enough  to  produce  interesting  effects. 

5.1  The  membrane  as  a  linear  filter 

That  the  neuron  membrane  behaves  like  a  filter  is  a  trivial  and  uncontroversial 
statement  because  whenever  a  current  is  applied  to  anything  a  voltage  will  result  and 
the  relation  between  them,  the  transfer  function  or  impedance,  will  be  a  filter  of  some 
sort.  Theoretical  and  empirical  studies  have  extended  the  utility  of  this  statement  by 
showing  how  the  filter  properties  arise  from  known  conductance  mechanisms,  and  by 
showing  in  many  cases  that  the  filter  will  be  linear  over  some  range  of  voltages. 

A  linear  filter  model  is  desirable  from  our  perspective  because  it  gives  a  simple, 
compact  and  empirically  discoverable  description  of  the  relation  between  input 
(current)  and  output  (voltage).  Most  theoretical  and  empirical  attention  has  been 
directed  toward  the  squid  giant  axon  ([Mauro,  1970  #882;  Poussart,  1977  #890;  Moore, 
1980  #886;  Sabah,  1969  #924;  Sabah,  1970  #928]).  Work  by  others  has  shown  that  the 
linear  filter  model  of  membrane  impedance  is  not  imiquely  valid  for  the  squid  giant 
axon  and  can  be  applied  to  a  variety  of  different  neurons  incorporating  a  variety  of 
different  conductances  (Calcium  conductance  in  thalamocortical  neurons,  trigeminal 
sensory  cells  under  anesthesia  [Puil,  1986  #883;  Puil,  1987  #884;  Puil,  1989  #885],  and 
NMDA  on  lamprey  spinal  neurons  [Moore,  1993  #889;  Moore,  1985  #842])  under 
different  conditions. 

Comparing  the  filter  properties  of  our  model  with  those  of  squid  giant  axon  can 
be  used  as  a  test  of  its  biological  validity.  The  only  apparent  discrepancy  between  the 
model  and  the  axon  is  a  slow  increase  in  phase  angle  of  the  transfer  function  of  the 
model  from  an  earlier  minimum  at  signal  frequencies  greater  than  300  Hz  which  is  not 


present  in  squid  giant  axon.  This  depends  on  the  size  of  the  integration  step  and  must 
be  an  artifact  of  the  numerical  methods.  This  presents  no  serious  problem  for  the  acute 
simulations  since  the  behavior  of  the  model  at  low  signal  frequencies,  which  is  what  we 
are  interested  in,  is  unaffected  by  the  choice  of  integration  step  size  within  the  range 
employed  in  the  remainder  of  the  simulations  used  in  these  studies  (0.05  ms  or  less). 

Otherwise  the  model  captures  the  functionally  important  characteristics  of  the 
squid  giant  axon.  As  a  filter,  the  squid  giant  axon  behaves  like  a  resonant  RLC  circuit. 
The  model  exhibits  the  same  characteristic  behavior.  Like  the  axon  the  resonance 
frequency  of  the  model  is  dependent  on  the  steady-state  resting  potential  maintained  by 
various  amounts  of  constant  injected  current  (Fig.  1).  As  expected,  hyperpolarizing  the 
cell  increases  the  damping  factor  and  shifts  the  resonant  frequency  to  lower  values 
while  depolarization  decreases  the  damping  factor  and  shifts  the  resonant  frequency  to 
higher  values. 

The  impedance  can  be  represented  in  a  variety  of  ways.  We  show  both  the 
impedance  plane  plot  in  the  frequency  domain  (Fig.  1),  and  the  IRF  in  the  time  domain 
(Fig.  2).  The  choice  of  representation  is  a  matter  of  convenience  since  they  are 
equivalent;  for  any  given  stimulus  current  waveform  they  both  predict  the  same  voltage 
waveform  response.  The  response  of  the  system  in  the  frequency  domain  is  computed 
by  multiplying  the  complex  impedance  by  the  transform  of  the  current  while  in  the  time 
domain  the  current  is  convolved  with  the  IRF.  Frequency-domain  analysis  is  better 
suited  for  empirically  estimating  the  impedance  and  calculating  the  response  to  a 
stimulus  in  real  cells  and  the  IRF  is  better  suited  for  describing  the  nature  of  the 
transformation  of  an  afferent  current  signal  into  a  membrane  potential  by  reactive 
membrane  conductances. 

The  duration  of  the  IRF  and  its  form  are  the  two  most  important  aspects  of  the 
filter  properties  of  the  membrane  for  what  follows.  The  extent  of  the  response,  the 
period  of  time  over  which  the  IRF  is  non-zero,  constitutes  a  type  of  memory  since  it 
indicates  how  long  the  effect  of  an  afferent  signals  persist  and  is  able  to  interact  with 
currents  that  arrive  at  other  times.  Where  the  IRF  is  zero  the  current  impulse  has  no 
influence  on  the  future  behavior  of  the  system.  Beyond  that  time  the  input  and  output 
are  causally  imcoupled.  Here,  the  response  to  a  single  small  impulse  lasting  less  than  a 
microsecond  persists  for  several  tens  of  milliseconds  and  the  nature  of  this  response 
depends  upon  the  resting  potential.  This  means  that  factors  affecting  the  membrane 
potential  will  be  able  to  affect  the  breadth  of  the  window  in  time  through  which  the 
neuron  observes  its  input.  In  the  same  way,  large  signals  may  have  the  same  effect  by 
placing  the 

In  the  case  of  the  squid  giant  axon,  the  membrane  exhibits  significant  resonance. 
This  means  that  the  membrane  potential  responds  to  a  current  impulse  with  a  damped 
oscillation.  Not  all  neurons  exhibit  resonance  and  in  the  squid  axon  it  can  be 


suppressed  by  hyperpolarization,  e.g,  [Mauro,  1970  #882;  Guttman,  1974  #914]. 
Resonance  imbues  the  temporal  relation  of  multiple  input  signals  with  an  importance 
foreign  to  a  passive  RC  membrane  where  the  interaction  between  two  events  simply 
diminishes  with  the  interval  of  time  separating  them.  With  resonance  there  is  also  a 
qualitative  change  in  the  response  over  time  where  the  interaction  of  an  impulse  with 
another  event  can  be  synergistic,  antagonistic  or  neutral,  depending  the  interval 
between  them.  The  phase  relation  or  temporal  separation  between  inputs  becomes  a 
potentially  significant  source  of  information  for  the  cell. 

The  relevant  properties  of  resonance  are  its  characteristic  frequency  and  its 
strength.  The  characteristic  or  resonant  frequency  of  the  filter  is  the  frequency  at  which 
the  magnitude  of  the  impedance  peaks  and  the  strength  of  the  resonance  can  be  judged 
by  the  size  and  damping  rate  of  the  oscillations  in  the  IRF.  This  frequency  depends 
most  heavily  on  the  characteristics  of  the  channels  involved  while  the  damping  factor 
depends  on  their  density.  The  resonant  frequency  of  the  squid  giant  axon  is 
approximately  linearly  proportional  to  the  membrane  potential  [Sabah,  1970  #928].  The 
relation  is  more  complex  in  other  cells  that  have  been  studied  but  it  usually  exhibits 
polarization-dependent  changes  in  one  or  both  of  these  properties.  Dorsal  spinal 
lamprey  neurons  [Moore,  1993  #889]  show  increased  resonance  and  higher  resonant 
frequencies  with  depolarization  as  do  trigeminal  sensory  neurons  [Puil,  1987  #884]. 
Neurons  of  the  nucleus  ovoidalis  of  the  chicken  [Strohmann,  1994  #932]  show  a 
decrease  in  resonance  away  from  the  normal  resting  potential  with  a  weak  increase  in 
resonant  frequency  with  depolarization.  In  neocortical  cells  the  resonant  frequency 
increases  with  hyperpolarization  [Hutcheon,  1994  #934].  In  guinea  pig  mediodorsal 
thalamic  neurons  [Puil,  1994  #935]  describe  a  minority  of  neurons  which  show 
decreased  damping  but  no  change  in  resonant  frequency  with  depolarization, 
preference 

Even  though  subthreshold  linear  impedances  appear  in  many  different  neurons 
there  is  a  reluctance  to  employ  them  in  describing  neural  function.  The  reason  for  this  is 
that  linearity  only  holds  for  small  signals  and  the  range  of  potentials  over  which  the 
filter  models  are  linear  is  narrow  compared  to  the  dynamic  range  of  operation  of  a 
neuron  [Koch,  1984  #922;  Koch,  1987  #680].  The  range  of  voltage  over  which  the 
impedance  of  the  squid  giant  axon  is  linear  is  between  1  and  2  mV  arormd  the  resting 
potential  or  about  3  mV  peak-to-peak  (p-p).  Using  subjective  judgment  excursions  of 
+/-  5  mV  from  the  resting  potential  have  been  reported  to  be  acceptable  in  squid  giant 
axon  [Sabah,  1969  #924;  Mauro,  1970  #882].  But  voltage-clamp  experiments  [Moore, 

1980  #886]  show  that  linearity  strictly  only  holds  within  0.27  mV  of  the  resting  potential 
where  current  responses  to  voltage  steps  were  symmetric.  Nonlinearity  increases 
beyond  that  point  but  does  not  become  significant  until  the  voltage  steps  are  greater 
than  1  mV  from  the  resting  potential.  This  is  supported  by  the  observation  that  the 


harmonic  distortion  increases  with  stimulus  amplitude  but  only  exceeds  10%  for 
stimulus  signals  with  peak-to-peak  amplitudes  above  3  mV.  In  addition,  analysis  of 
linearized  models  of  the  Hodgkin-Huxley  equations  become  imstable  (fail)  at  a  critical 
voltage  of  5.35  mV  [Sabah,  1970  #928;  Koch,  1984  #922]  above  the  resting  membrane 
potential.  Thus,  four  different  criteria  (stability  analysis,  response  symmetry,  harmonic 
distortion  and  subjective  judgment)  concur  that  the  linear  range  of  the  filter  function  in 
squid  giant  axon  is  not  more  than  5  mV  from  resting  potential. 

Despite  the  fact  that  linearity  is  strictly  valid  for  only  a  narrow  portion  of  the  full 
dynamic  range,  the  errors  introduced  by  the  failure  of  linearity  are  often  relatively  mild. 
This  means  that  the  linear  approximation  that  is  valid  near  the  resting  potential  can 
provide  important,  semi-quantitative  information  about  the  behavior  of  the  neuron 
beyond  the  range  of  acceptable  linearity.  For  example,  estimates  of  the  squid  giant  axon 
admittance  obtained  with  2  mV  p-p  (linearly  valid  range)  and  8  mV  p-p  (linearly  invalid 
range)  perturbations  differ  only  in  the  signal  to  noise  ratio  of  the  estimated  transfer 
function  [Poussart,  1977  #890].  Using  control  voltage  signals  outside  the  valid  linear 
range  did  not  change  the  estimated  characteristic  frequency  or  other  gross 
characteristics.  Similarly,  Guttman,  Feldman  and  Lecar  [,  1974  #914]  reported  only  mild 
deviations  from  linearity  as  the  amplitude  of  the  white  noise  stimulus  used  to  measure 
the  squid  axon  filter  function  was  increases  past  threshold,  more  than  10  mV  above  the 
resting  potential.  The  principle  effect  of  larger  noise  amplitude  was  a  decrease  in 
damping.  The  characteristic  frequency  was  only  slightly  affected. 

The  studies  looking  at  the  range  of  linearity  mostly  used  square  current  or 
voltage  steps.  Random  or  pseudo-random  stimulus  waveforms  were  used  in  the 
studies  that  reported  only  minor  deviations  from  linearity  over  large  voltage  ranges. 

An  obvious  difference  between  the  two  methods  of  stimulation  is  that  with  noise  a 
variety  of  membrane  potentials  were  visited  briefly  while  the  mean  voltage  remained 
constant  but  with  constant  current  or  voltage  steps  the  membrane  evolves  from  one 
steady  state  voltage-dependent  impedance  to  another.  Under  these  circumstances 
nonlinearities  are  apt  to  be  more  salient. 

This  suggests  that  as  long  as  membrane  potential  excursions  are  relatively  brief 
with  respect  to  the  settling  time  of  the  membrane  conductances,  the  linear  behavior  near 
the  resting  potential  will  continue  to  be  useful  over  wider  ranges  of  voltages  than  the 
limits  of  strict  steady-state  linearity  suggest.  This  conjecture  is  consistent  with  the  ACT 
shown  here  (Fig.  4).  Depolarization  increases  the  resonant  frequency  of  the  membrane, 
but  the  timing  of  the  phases  of  the  ACT  are  closer  to  the  impedance  at  the  resting 
potential  than  depolarized  potentials  near  threshold. 

The  main  difference  between  the  IRF,  which  describes  the  behavior  of  the 
membrane  over  a  linear  range  of  potentials,  and  the  ACT,  which  describes  it  over  a 
nonlinear  range  of  potentials,  is  the  flattening  of  the  early  phase  of  the  IRF.  The  cross 


correlation  function  between  white  current  noise  and  membrane  potential,  a  similar 
statistic  to  the  IRF,  behaves  the  same  way  [Guttman,  1974  #914];  as  the  intensity  of  the 
noise  is  increased  the  cross  correlation  function  evolves  smoothly  from  a  form  like  the 
IRF  to  one  like  the  ACT.  Theirs  is  the  only  study  we  know  of  that  has  explored  a 
continuum  between  subthreshold  and  suprathreshold  membrane  behavior.  Although 
Bryant  and  Segundo  [,  1976  #815]  obtained  an  ACT  showing  many  of  these  features, 
from  which  they  proposed  a  theory  of  spike  train  genesis  based  on  a  linear  filter  model, 
they  did  not  try  to  relate  that  filter  to  membrane  properties  or  to  sub  threshold  filter 
behavior. 

In  significantly  different  experiments  Bialek  and  coworkers  [,  1991  #870]  looked 
at  the  relation  of  spike  times  in  the  HI  neuron  of  the  blowfly  which  receive  visual 
information  that  is  gathered  from  several  independent  receptors  and  has  passed 
through  several  synapses.  Despite  the  dispersed  and  indirect  access  to  visual 
information,  the  acceleration  of  the  visual  field  can  be  read  accurately  from  the  HI  spike 
train  by  applying  a  simple  linear  filter  similar  to  the  ACT  described  here  and  by  others. 
The  major  difference  is  that  they  make  no  reference  to  biophysical  mechanisms  and  the 
filter  they  extract  from  the  spike  train  represents  a  transfer  function  across  the  entire 
visual  system.  In  their  theoretical  analysis  Bialek  and  Zee  [,  1990  #936]  show  how 
adaptive  linear  filters  have  advantages  for  optimally  balancing  conflicting  operational 
constrains  in  sensory  systems  and  suggest  that  linear  filtering  is  an  important  aspect  of 
information  processing  in  nervous  systems  in  general  [Bialek,  1993  #915]. 

5.2  Determinism  in  membrane  filter  function 

For  a  scheme  like  that  proposed  by  Bialek  to  work,  precise  phase  (timing) 
information  must  be  preserved  through  the  synapses  in  the  network.  The  notion  that  at 
the  level  of  a  whole  cell  the  dynamics  are  mostly  deterministic  suggests  that  this  should 
be  the  case  and  the  paper  by  Bryant  and  Segundo  [,  1976  #815]  show  the  ability  of  single 
neurons  to  respond  consistently  to  complex  stimulus  patterns.  The  spikes  usually 
appear  at  the  same  places  in  the  course  of  repeated  presentations  of  a  single  noise 
sample,  and  when  this  does  not  happen,  they  appear  at  favored  alternative  times.  This 
observation  has  been  corroborated  more  recently  by  Mainen  and  Sejnowski  [Mainen, 
1994  #937]  and  Bair,  Koch,  Newsome,  and  Britten  [,  1994  #938]. 

Three  conclusions  may  be  drawn  from  these  research  threads.  First,  as  evidence 
accumulates  that  the  information  in  neuronal  spike-trains  is  not  simply  a  mean  firing 
rate  code  (as  reviewed  in  the  introduction),  then  we  must  accept  that  the  afferent  stream 
is  imlikely  to  be  random  noise.  Non-random  structure  in  both  input  and  output  should 
be  expected.  Second,  although  the  background  afferent  stream  may  be  unpredictable 
and  of  a  small  magnitude,  there  are  reasons  to  believe  it  is  deterministically  related  to 


aspects  of  the  environment,  and  that  its  shape  over  time  is  a  significant  source  of 
information  for  neurons.  Third,  in  order  for  there  to  be  significant  information  in  the 
details  of  the  structure  or  timing  of  the  input  there  has  to  be  a  precise  and  exact  relation 
between  input  and  output  such  that  small  details  are  mapped  sensitively.  The 
transforming  computations  of  the  membrane  must  reliably  preserve  details.  Stated 
within  the  context  of  our  findings,  the  linear  perturbations  must  be  reliably  or 
deterministically  transformed  by  the  nonlinear  behavior  of  the  membrane.  The 
impedance  history  of  the  membrane,  as  expressed  variously  in  the  IRF,  ACT,  CACTs, 
and  the  isopleths  indicates  that  this  transform  may  be  quite  complex,  and  that  features 
of  the  form  of  the  input  may  dissipate  or  be  lost. 

Because  deterministic  systems  retain  information  about  small  details  it  is 
important  to  consider  the  significance  of  small,  subthreshold  signals.  Some  synaptic 
events  are  unquestionably  large,  important  and  reasonably  well  known.  They  have 
attracted  a  large  amount  of  the  attention  of  researchers  over  the  years.  But  there  are 
also  many  instances  where  there  is  an  on-going  low-level  sub-threshold  stream  of 
afferent  activity.  Evidence  that  much  synaptic  activity  is  of  small  magnitude  and  can  be 
foimd  scattered  throughout  the  literature  even  though  studies  of  low-level  background 
afferent  activity  is  scarce.  For  example,  [Fetz,  1983  #939]  report  monosynaptic  potentials 
between  0.15  and  3.1  mV  (mean  0.75  mV)  in  cat  motor  neurons.  In  rat  visual  cortex 
[Mason,  1991  #940]  show  a  range  of  synaptic  potentials  of  0.05  to  0.5  mV.  The 
background  synaptic  activity  in  chicken  nucleus  ovoidalis  neurons  has  a  p-p  magnitude 
of  about  3  mV  [Strohmann,  1994  #932].  We  have  shown  here  that  small  subliminal 
currents  have  a  significant  effect  on  the  outcome  of  superthreshold  events  (Figures  6- 
10). 

The  deterministic  Hodgkin  and  Huxley  model  tells  us  that  this  is  inevitable.  Is  it 
important?  Yes.  In  the  presence  of  a  fixed  synaptic  event  the  shape  of  the  ACT  by  itself 
provides  little  information  about  the  structure  of  the  data  stream  in  the  input  circuit. 
The  recoverable  information  about  the  noise  stream  is  determined  by  the  bandpass 
characteristics  of  the  resting  impedance  which  translates  the  local  "shape"  of  the 
stimulus  into  changes  in  the  timing  of  action  potentials  from  unperturbed  values.  This 
necessitates  consideration  of  the  activity  of  the  cell  within  the  circuit.  The  timing  of  the 
action  potential  can  have  no  meaning  without  defining  a  frame  of  reference.  The  frame 
of  reference  is  provided  by  using  events  in  other  members  of  the  circuit  as  reference 
points,  where  the  spike  time  is  defined  as  the  interval  between  the  pre-  and  post- 
synaptic  action  potentials  as  a  cross-correlation  function.  The  model  used  here  is  a 
simple  circuit  where  there  are  two  distinct  and  independent  information  sources,  a 
synaptic  event  and  an  independent  subthreshold  source  which  is  either  a  random 
stream  to  obtain  the  modified- ACTs  or  CACTs  in  Figs.  6-8,  or  a  deliberately  placed  but 
sill  independent  impulse  to  obtain  the  isopleths  in  Figs.  9-10.  In  a  sense,  the 


suprathreshold  synaptic  event  provides  an  observational  anchor  with  which  either  the 
input  data  stream  or  the  output  action-potential  latencies  may  be  analyzed. 

When  this  information  is  used  the  spike  timing  is  seen  to  contain  significant 
information  about  the  shape  of  the  subthreshold  afferent  channel  before  and 
immediately  preceding  the  EPSC  and  that  this  information  is  a  faithful  scaled  version  of 
the  filter  function  which  translates  magnitude  to  postsynaptic  spike  latency. 

Comparing  the  CACTs  obtained  from  the  afferent-  (Fig.  7)  and  efferent-  (Fig.  8)  frames 
of  reference  shows  that  as  information  is  obtained  about  the  source,  the  information 
obtained  from  an  observation  increases.  Observations  bound  only  to  the  efferent  frame 
of  reference  are  ignorant  of  the  events  yielding  no  action  potential  and  this  excludes  an 
entire  class  of  CACT  from  consideration. 

In  this  situation  knowledge  of  the  interval  between  the  action  potential  and 
synaptic  event  implies  knowledge  of  the  noise  channel  in  the  vicinity  of  the  FPSC.  The 
limit  to  the  amount  of  obtainable  information  here  is  the  precision  of  the  measurement 
of  the  postsynaptic  spike  timing.  The  CACTs  consists  of  a  smooth,  graded  continuum 
so  that  as  the  precision  of  the  timing  of  the  spike  increases  there  will  be  a  corresponding 
increase  in  the  precision  with  which  the  CACTs  are  known.  Residual  ignorance  can  be 
shifted  between  action  potential  timing  and  EPSC  form,  but  the  correlation  with  the 
noise  remains  constant  except  for  the  temporal  offsets  associated  with  the  latencies  of 
each  class.  The  offset  in  the  CACT  from  the  class  of  prolonged  latencies  at  the  tail  of  the 
histogram  show  that  information  about  the  noise  channel  during  the  EPSC  can  be 
preserved  over  several  milliseconds  and  is  immune  to  noise  after  the  EPSC. 

To  understand  how  these  principles  can  be  applied  in  real  situations  requires 
measurements  of  both  input  and  output  in  order  to  ascertain  how  different  components 
in  a  network  cooperate.  In  the  case  of  the  blowfly,  for  example,  [Bialek,  1991  #870],  is 
the  observed  filter  function  a  result  of  prefiltering  by  visual  elements  or  is  it  a  fairly 
direct  reflection  of  the  local  electrical  properties  of  the  neuron?  If  linear  filters  are 
generally  appropriate  then  the  convolution  of  the  transfer  function  across  HI  (the  ACT), 
with  the  transfer  function  between  the  visual  stimulation  and  the  afferent  current 
should  yield  the  overall  transfer  function  already  described. 

5.3  Degeneracy 

The  consequences  of  the  deterministic  nature  of  the  Hodgkin-Huxley  model 
raises  interesting  problems.  From  a  dynamical-  theoretical  perspective  [Takens,  1981 
#54;  Packard,  1980  #43],  the  state  of  the  membrane  can  be  "resurrected"  from  a  time  - 
series  of  measurements  of  a  single  variable,  which  in  typical  neurophysiological 
experiments  is  usually  the  membrane  potential.  The  system  of  coupled  differential 
equations  is  internally  consistent  and  can  be  used  to  transmit  information  from  one  cell 


to  another  through  continuous  observation  of  only  one  of  the  variables.  This  may  be 
possible  in  neural  networks  in  which  transmission  is  relatively  faithfully  accomplished 
by  graded  transmitter  release,  as  has  been  observed  in  many  experimental  systems  since 
there  is  a  one-to-one  mapping  of  an  input  state  with  an  output  state  [Anderson,  1981 
#946;  Blight,  1980  #951;  Davis,  1989  #947;  Fain,  1977  #950;  Graubard,  1983  #729; 
Mendelson,  1971  #948;  Pearson,  1975  #952;  Shaw,  1972  #949;  Siegler,  1984  #944; 

Werblin,  1969  #629;  Maynard,  1975  #942;  Roberts,  1981  #945], 

In  the  large  class  of  neurons  that  communicate  with  one  another  via  action 
potentials  the  output  of  a  neuron  is  discontinuous  and  so  represents  a  significant 
transformation  of  its  input  signal,  and  is  subject  to  degeneracies.  In  following  papers, 
we  shall  examine  some  of  these  transformations  [Edstrom,  1994  #912;  Mpitsos,  1994 
#895].  The  foundation  of  this  work  is  presented  there.  On  the  one  hand,  because  the 
Hodgkin-Huxley  equations  are  deterministic,  we  expect  that  each  unique  stimulus  will 
produce  a  tmique  response.  This  is  indeed  the  case  when  the  full  four-state  description 
is  considered  or  even  when  a  single  state  variable  is  monitored  continuously.  But,  the 
information  in  spike  timing  alone  is  not  complete.  Consider  Fig.  10  which  shows  the 
i/o  curves  for  several  different  current  impulses  presented  at  a  variety  of  different 
times.  A  horizontal  line  parallel  to  the  x-axis  will  intersect  a  number  of  isopleths,  many 
will  be  intersected  two  or  more  times.  Therefore  a  given  impulse  can  produce  the  same 
change  in  latency  when  presented  at  two  different  times.  Worse,  for  any  given  latency 
change  there  are  infinitely  many  combinations  of  impulse  amplitude/presentation  time 
which  will  produce  that  particular  latency. 

Thus,  even  the  simple  network  that  we  have  used  here  has  the  potential  of 
producing  a  highly  degenerate,  infinitely  many-to-one  mapping  of  an  input  data  stream 
into  an  output  stream.  A  given  latency  may  arise  from  any  number  of  combinations  of 
different  impulse  magnitudes  and  presentation  times.  An  impulse  having  a  given 
amplituded  and  presentation  time  is  correlated  produces  a  vmque(V,m,h,n)  state  state  of 
the  membrane;  e.g,  compare  the  states  for  points  a,c,e  'm  Fig.  11  that  all  represent  the 
same  latency.  But  information  about  these  states  is  not  contained  in  a  spike  train 
because  observing  only  the  moment  when  the  voltage  crosses  a  detection  threshold 
collapses  the  full  state-space  trajectory  onto  a  lower,  one-dimensional  point.  Thus,  a 
neuron  cannot  transmit  complete  information  about  its  internal  state  by  spike  time 
alone.  This  raises  questions  about  how  the  spike  time  is  "meant"  to  be  interpreted. 

In  principle,  the  shift  from  graded  neuronal  connections,  which  are  amenable  to 
detailed  phase-space  analyses  of  their  dynamics,  to  spiking  communication  between 
neurons  is  presumed  to  be  theoretically  possible  [Wang,  1992  #732].  Studies  using 
model  systems  and  integrate-and-fire  methods  have  addressed  the  problem  of  whether 
there  is  a  deterministic  relationship  between  the  underlying  continuous  dynamics  and 
threshold-generated  point  processes  [Longtin,  1991  #1014;Preissl,  1990  #587;Sauer,  1994 


#963].  Application  of  a  number  of  methods  to  biological  neural  systems  have  obtained 
evidence  both  for  deterministic  and  nondeterministic  activity  [Longtin,  1990  #1015; 
Schiff,  1994  #960;  Schiff,  1994  #990].  Our  aim  here  has  been  to  understand  the 
input/ output  transformation  process  in  its  simplest  expression  as  occurs  through  the 
history  of  membrane  changes  as  in  a  simple  two-cell  network.  At  this  level  of 
organization,  our  findings  indicate  that  there  is  considerable  loss  of  information  about 
the  input  stream.  Moreover,  the  findings  on  CACTs  (Figs.  6-8)  show  that  there  is  loss  of 
information  whether  an  observer  looks  from  the  perspective  of  the  input  or  the  output 
data  streams  on  cell-2.  Alternatively,  the  information  may  not  be  necessary,  and 
degeneracy  may  be  necessary  since  it  gives  the  network  enough  degrees  of  freedom  to 
construct  a  desired  particular  output  from  a  variety  of  inputs,  as  circumstances  dictate 
[Longtin,  1990  #1015;  Longtin,  1991  #1014;Longtin,  1991  #1013;Mpitsos,  1986  #38; 
Bulsara,  1991  #1016]. 

In  any  event,  this  scenario  is  what  real-life  neurons  may  have  to  work  with,  or 
must  have  evolved  membrane  mechanisms  or  network  functions  that  adjust  to  it.  Here 
we  have  defined  the  first  components  of  such  a  scenario  in  the  linear  filter  dynamics  of 
simple  membrane.  In  specific  cases,  biological  systems  have  devised  ways  to 
accommodate  or  adjust  for  such  degeneracies,  as  shown,  for  example,  by  the  fidelity  of 
information  transfer  in  studies  of  linear  models  of  blowfly  [Bialek,  1991  #870;  Glantz, 
1994  #919]  and  crayfish  visual  systems.  However,  what  we  know  formally  of  dynamical 
systems  dynamics  has  come  largely  from  studies  of  differential  and  difference 
equations.  The  application  of  such  formalism  to  account  for  the  dynamics  of  the  simple 
networks  described  here  will  be  quite  difficult,  if  possible  at  all.  Subsequent  papers  will 
deal  with  these  networks  numerically  to  determine  how  the  linear  filter  dynamics  may 
lead  both  to  stable  and  variable  firing  patterns.  The  overall  aim  will  be  to  ascertain 
what  information  of  the  input  is  retained  in  the  output  of  the  neuron,  what  is  lost  or 
dissipated,  and  how  the  characteristics  of  neurons  and  of  the  network  architecture 
accommodate  to  such  loses  or  are  able  to  function  accurately  in  the  face  of  unresolved 
degeneracies. 
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FIGURE  LEGENDS 


FIGURE  1:  White  noise  analysis  of  complex  impedance  Z.  A:  Comparison  of  impedance- 
plane  plots  of  the  normal/  depolarized  and  hyperpolarized  membrane  with  0.05  msec  time 
step  at  three  membrane  potentials.  The  open  circles  designate  data  from  the  normal  resting 
potential  {£?n=  ■  63.32  mV);  filled  triangles  are  for  the  hyperpolarized  membrane  (E^  =  -66.40 
mV  using  sustained  current  of  -9.5  x  lO'^l  Amp)  and  open  triangles  is  for  depolarized 
membrane  {Em  =  -60.208  mV  using  sustained  current  of  13  x  lO'^^  Amp).  White  noise  current 
was  subthreshold  for  spike  genesis  in  all  cases.  B:  The  vector  length  or  magnitude  of  Z  as  a 
function  of  frequency.  Filled  and  open  circles  are  for  the  data  presented  in  (C)  at  normal 
membrane  polarization.  C:  The  vector  Z  as  a  plot  of  reactance  vs.  resistance  for  two 
integration  step  sizes  at  the  normal  membrane  potential.  Open  circles  are  for  step  =  0.05 
msec;  filled  circles  are  for  step  =  0.10  msec.  D;  The  phase  of  Z  (vector  angle  in  (A  and  C))  as  a 
function  of  frequency.  Symbols  are  as  in  (A-C). 

At  the  normal  resting  potential  there  is  a  significant  resonant  peak  at  about  40  Hz  that 
is  shifted  up  to  55  Hz  when  the  membrane  is  depolarized  and  down  to  30  Hz  when  it  is 
hyperpolarized.  Above  300  Hz  the  modeling  process  introduces  some  phase  distortions.  In  D 
the  phase  reaches  a  minimum  around  300  Hz  and  then  angles  back  up.  This  distortion  is 
more  severe  for  the  simulation  using  a  0.1  msec  time  step  than  for  the  simulation  using  a  0.05 
msec  time  step.  There  is  no  apparent  difference  between  the  two  at  frequencies  below  300 
Hz. 

Typical  Current  (all  cases;  example  is  from  0.05  msec  time  step  simulation/  N  =  16384): 


Mean:  1.5493  x  lO'^^  Amps. 

Tvpical  Voltage  responses: 

SD:  4.1111  X  10‘13  Amps. 

Max: 

Min: 

1.5443  X  10'12  Amps. 
-1.4915  X  10*12  Amps. 

Mean  (normal): 

-0.063319  Volts. 

SD: 

7.9827  X  10-6  Volts. 

Max: 

-0.063293  Volts. 

Min: 

-0.063342  Volts. 

Mean  (depolarized): 

-0.060208  Volts. 

SD: 

8.0384  X  10-6  Volts. 

Max: 

-0.060182  Volts. 

Min; 

-0.060232  Volts. 

Mean  (hyperpolarized): 

-0.066403  Volts. 

SD: 

7.3067  X  10-6  Volts. 

Max: 

-0.066378  Volts. 

Min: 

-0.066425  Volts. 

The  cell  was  injected  with  white  noise  for  0.8192  sec  in  all  cases  (8192  points  @  0.1  msec 
integration  steps;  and  16384  points  @  0.05  msec  integrations)  so  that  the  minimum  resolvable 
frequency  step  is  1.2207  Hz.  The  Nyquist  frequency  was  10  kHz  with  0.05  msec  and  5  kHz 
with  0.1  msec  time  steps. 
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FIGURE  2:  Time  course  of  the  impulse  response  function  (IRF):  back  transformation  of  the 
complex  impedances  shown  in  Fig.  1.  Circles  show  every  10th  point  in  a  five-point  center 
weighted  running  mean  of  the  IRF  from  normal.  The  running  average  is  needed  to  suppress 
high  frequency  noise,  remnants  of  which  can  be  seen  on  the  two  curves,  resulting  from  phase 
distortions  at  the  high  frequency  end  of  the  spectrum. 

FIGURE  3:  Noise-activated  action  potentials.  Two  time  scales  are  shown.  The  top  trace  is  an 
expansion  of  the  region  in  the  lower  trace  noted  by  the  dashed  lines,  and  shows  example  of 
spike  intervals  that  were  excluded  from  the  computation  of  the  ACT.  Current  was  injected 
continuously.  Mean  noise:  -1.676  x  10'^^  Amp;  SD  =  3.73  x  10*^®  Amp,  Time  step  =  0.05  msec. 

FIGURE  4:  Time  course  of  the  ACT.  Vertical  axis:  average  current.  Horizontal  axis:  time 
preceding  the  occurrence  action  potentials;  N  =  30,000.  Mean  current  Mean  noise:  -1.676  x  10" 
13  Amp;  SD  =  3.73  x  lO'l^l  Amp.  Dashed  horizontal  lines:  96%  Confidence  Limit,  CL  = 

=  Mean  ±  4.3  x  10‘12  Amp.  The  analysis  was  on  the  same  time  series  as  shown  by  the  excerpt 
in  Fig.  4.  This  shows  clearly  that  the  average  stimulating  membrane  current  has  a  complex 
three-component  waveform  lasting  approximately  30  msec  before  the  occurrence  of  the 
action  potential.  The  first  component  (0  to  5  msec  preceding  the  action  potential)  is 
depolarizing.  The  second  (5  to  20  msec)  is  hyperpolarizing,  and  the  third,  just  barely  above 
the  96%  confidence  limit,  is  depolarizing.  Since  the  autocorrelation  function  of  the  injected 
current  is  flat,  except  for  the  first  7  points,  these  correlations  between  the  action  potentials 
and  their  preceding  history  of  membrane  currents  are  introduced  by  the  membrane.  The  time 
course  of  the  first  two  components  is  identical  to  the  time  course  of  the  two  components 
obtained  in  the  IRF.  The  third  is  only  slightly  depolarizing,  and  is  not  observed  in  the  IRF. 

FIGURE  5:  Effects  produced  by  the  three  phases  of  the  ACT.  The  waveshape  and  amplitude 
of  the  three  phases  of  the  ACT  were  used  as  stimuli  for  injecting  current  into  the  cell.  A: 
Membrane  potential.  B:  Full  ACT  wave  shape.  C:  ACT  with  phase  three  removed.  D:  Phase 
one  of  the  ACT.  Thick  line  in  the  voltage  trace  is  the  response  to  phase  three.  Thin  line  is  the 
combined  responses  to  the  full  ACT  and  the  ACT  less  phase  three. 

FIGURE  6.  Modified  ACTs:  In  the  next  three  illustrations,  EPSC-evoked  action  potentials  wiU 
be  used  to  examine  the  effect  of  subthreshold  noise  stimulation.  A:  Low  and  high  noise 
modified  ACTs  in  which  random  stimulation  is  paired  with  an  EPSC.  B:  Synaptic  ACTs. 
60,000,  35-ms  simulation  sweeps  were  conducted  in  which  random  noise  perturbation  was 
paired  with  an  EPSC  in  each  sweep.  Since  the  synaptic  ACT  is  calculated  using  the  timing  of 
the  action  potential,  the  preceeding  currents  of  the  EPSC  are  shifted  from  one  sweep  to  the 


next,  causing  a  slurring  of  the  synaptic  currents.  C  and  D;  Frequency  histograms  of  latencies 
obtained  with  low  and  high  noise  levels.  The  integration  step  size  was  0.001  msec.  In  order 
to  show  an  expanded  scale,  the  traces  here,  and  in  Figs.  7  and  8,  are  of  1500  integration-step 
excerpts.  In  each  sweep  a  different  seed  was  used  to  generate  pulses  having  random 
amplitude  variation  (same  parameters  as  used  in  Fig.  5).  At  25  ms  into  each  sweep  a  single 
EPSC  was  given  that  had  Ggyn  =  3  x  10-8  Siemens.  Each  EPSC  generated  and  action  potential 
whose  latency  was  defined  using  the  start  of  the  sweep  as  time  zero. 

FIGURE  7:  Average  currents  can  be  calculated  from  the  input-stream  frame  of  reference 
using  modified- ACTS  contingently.  The  latencies  used  in  the  low-  and  high-noise 
experiments  of  Fig.  6  were  placed  into  5  histogram  bins  having  equal  occupancy.  Each  range 
of  latencies  provided  the  contingency  for  selecting  the  particular  noise-sweeps  to  use  in  the 
averages.  A  and  C:  Modified-contingent  ACTs  (modified-CACTs)  for  low  and  high  noise. 
All  are  scaled  selectively  for  each  latency  bin.  Trajectories  producing  the  longest  latencies 
dip  most  negatively  near  the  EPSC,  and  those  generating  the  shorter  latencies  deflect  more 
positively.  B  and  D:  Synaptic  C ACTs  for  all  latency  are  equivalent  to  the  EPSC.  Averages 
obtained  using  only  the  output  frame  of  reference  (Figs.  5  and  6)  lose  information  about  the 
input  perturbations  and  of  timing  of  the  EPSC.  This  information  was  uncovered  by  forcing 
the  data  into  the  histogram.  This  information  is  lost  to  an  observer  who  has  information  only 
about  the  spike  intervals  generated  in  the  postsynaptic  neuron. 

FIGURE  8.  Observing  spike  latencies  from  the  frame  of  reference  of  the  output  data-stream 
using  modified-CACTs.  Same  histogram  data  as  used  to  construct  Figs.  6  and  7.  A: 
Modified-CACTs  constructed  using  the  action  potentials  as  the  reference  point.  B:  Synaptic 
ACTs  corresponding  to  each  trajectory  in  (A);  the  numbers  used  to  identify  the  synaptic 
trajectory  progress  serially  and  are  matched  with  the  numbers  in  (A).  C  and  D:  Same  as  in  (A 
and  D)  but  with  a  higher  level  of  random  noise.  The  relative  timing  between  the  classes  of 
spike  latencies  and  the  corresponding  noise  currents  is  retained.  But  information  is  lost 
between  the  timing  of  the  occurrence  of  the  EPSC  and  the  action  potential. 

FIGURE  9.  Weak  preconditioning  current  pulses  affect  EPSC-evoked  action  potential 
latencies  after  membrane  potential  changes  occurring  in  response  to  the  pulses  have  almost 
died  out.  A:  Preconditioning  current  pulse  duration:  0.010  msec,  given  at  5  msec.  Dashed 
line:  hyperpolarizing  pulse.  Dotted  line  no  preconditioning  pulse.  Thick-solid  line: 
depolarizing  pulse.  Note  that  the  membrane  potential  has  recovered  before  the  onset  of  the 
EPSC  which  occurs  at  11.16  msec.  B:  Same  as  (A)  but  at  different  scales,  showing  the  EPSC 
and  the  action  potentials;  dotted,  dashed,  and  thick-line  traces  are  for  the  same  conditions  as 
in  (A).  C:  Membrane  response  to  a  hyperpolarizing  pulse;  EPSC  onset  is  indicated  by  the 
thin  vertical  line.  D  and  E:  Time  course  of  changes  in  the  potassium  gate  n  and  sodium  gate 
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m  during  the  voltage  time  course  shown  in  (C).  Magnitude  of  the  current  pulses  (±)  1  x  10*9 
Amp.  Integration  time  step  =  0.010  msec. 

FIGURE  10:  The  timing  of  single  preconditioning  current  pulses  produces  complex  changes 
in  action  potential  latencies.  In  each  40  msec  sweep  of  the  simulation  a  single  pulse  was  given 
before  the  onset  of  the  EPSC.  A:  Onset  (24  msec)  and  time  course  of  the  EPSC.  B:  Action 
potential  produced  by  the  EPSC  in  the  absence  of  a  preconditioning  current  pulse.  C  and  D: 
latencies  of  action  potentials  produced  in  hyperpolarized  and  normal  membrane  to 
depolarizing  (solid  curves)  and  hyperpolarizing  (dotted)  current  pulses;  hyperpolarization 
used  here  is  the  same  as  in  Figs.  1  and  3.  Each  curve  was  obtained  using  a  different 
amplitude  of  the  current  pulse.  The  incomplete  lines  in  some  of  the  negative  current 
isopleths  indicate  suppression  of  the  EPSC-evoked  action  potential  at  those  current/time 
combinations.  The  interval  between  the  points  on  the  curves  is  250  msec.  The  amplitude  for 
the  five  0.010  ms  depolarizing  and  hyperpolarizing  current  pulses  was,  in  multiples  of  lO'i^ 
Amp:  (±)  2, 4, 6, 8, 10.  Ggyn  =  3  x  10*8  Siemens  at  the  normal  resting  potential  and  4.0536  x  10* 
8  Siemens  at  the  hyperpolarized  potential,  a  value  chosen  by  trial  and  error  to  give  the  same 
vmperturbed  action  potential  latency  as  the  normal  control  EPSC.  Integration  time  step  = 
0.010  ms. 

FIGURE  11.  Relationship  of  i/o  curves  to  the  m  and  n  gate  dynamics.  A:  Two  i/ o 
curves  or  isopleths  obtained  using  ±  10  x  10*10  Amp  pulses.  Labeled  points:  a,  the 
normal  unperturbed  resting  state;  b,  the  point  of  maximum  action  potential  advance  at 
19  ms;  c,  the  null  point  of  no  effect  (22.44  ms);  d,  the  point  on  the  zero  isopleth 
corresponding  to  the  pulse-presentation  time  that  yields  the  maximum  retardation  (26 
ms;  in  this  case,  the  action  potential  is  suppressed  entirely  and  the  point  d  does  not  exist 
on  the  -10  x  10*1®  isopleth);  and  e,  the  terminal  point  of  no  effect  that  corresponds  to  the 
moment  the  membrane  potential  crosses  the  criterion  threshold  (-30  mV).  In  the  absence 
of  perturbing  influences  the  action  potential  crosses  that  threshold  at  30.96  ms.  The 
EPSC  begins  at  25  ms.  B:  Medium  close-up  of  the  m-n  phase  plane.  Trajectories  to  the 
left  of  the  control  (dotted)  trajectory  are  advanced  (reduced  latency)  and  the  trajectories 
to  its  right  are  retarded  (increased  latencies).  At  this  magmfication  the  state  of  the 
membrane  is  roughly  the  same  at  points  ci,  b,  and  c.  The  state  at  point  d  (26  ms  into  the 
simulation)  is  different  than  at  a,  b,  or  c  because  the  EPSC  precedes  it.  Point  e  labels  the 
m-n  state  when  the  membrane  crosses  the  spike-detection  threshold  (-30  mV).  C:  Initial 
section  of  the  trajectories  in  (B).  D:  Extreme  close-up  of  the  foot  of  the  trajectories  in  (B) 
where  points  fl,  b,  c  are  also  the  same  as  those  leveled  in(B).  Arcflcfcis  the  trajectory  of 
m-n  states  that  are  produced  by  a  -10  x  10*^®  Amp  pulse  when  no  EPSC  is  given. 
Trajectories  are  also  shown  for  the  m-n  states  that  emerge  in  time  for  action  potentials 
that  are  most  advanced  (solid  line  emerging  upward  from  point  b)  and  for  a  control 


trajectory  in  which  the  EPSC  is  not  paired  with  a  conditioning  pulse  (dots  emerging 
from  point  a).  Point  a  is  also  the  take-off  point  for  the  trajectory  relating  to  action 
potential  latencies  that  are  most  delayed(dashes  overlying  the  dots).  This  trajectory 
diverges  from  the  control  when  the  conditioning  pulse  is  presented  at  26  ms  into  the 
simulation,  1  ms  after  the  onset  of  the  EPSC.  When  using  paired  pulse/EPSC 
stimulation,  trajectories  originate  along  the  baseline  arc,  starting  successively  from 
point  a  (25  ms  before  the  EPSC)  and  progress  toward  point  b  . 

FIGURE  12.  Stretching  of  m-n  phase  space  in  time  after  the  onset  of  the  EPSC.  A:  Base 
arc  of  open-circle  points  are  take-off  positions  on  the  ach  arc  shown  in  Fig.  lOD  for  a 
selected  number  of  trajectories;  this  arc  represents  the  state  of  the  membrane  just  before 
the  onset  of  the  EPSC.  Dashed  line:  trajectory  emerging  from  point  c.  Solid  line: 
trajectory  emerging  from  point  b.  Dashed  line:  trajectory  emerging  from  point  a.  A-D: 
Time-slices  at  selected  times  after  the  onset  of  the  EPSC.  The  range  of  phase  space 
dimensions  are  the  same  in  all  panels  in  order  to  show  the  stretching  of  the  loops  in 
time.  In  panel  B,  the  time  slices  have  been  taken  just  after  the  separatrix  (see  Fig.  12)  of 
ni-n  states  that  are  above  and  below  the  level  for  generating  action  potentials.  Thus,  the 
bottom-most  solid  line  now  represents  the  trajectory  of  the  most  retarded  action 
potential  latency  (here  the  spike  is  aborted),  and  the  control  trajectory  is  now  shown  by 
the  point  lying  between  the  two  solid  lines. 

FIGURE  13.  Separatrix  between  phase-space  states  for  spiking  and  nonspiking  membrane.  A: 
Membrane  potential  obtained  using  a  0.001  ms  depolarizing  pulse  that  way  just 
suprathreshold  ( 512.3152366087  x  lO'^^  Amp)  and  subthreshold  (512.3152366086  x  lO'l^ 
Amp)  for  generating  an  actin  potential.  B:  Phase  space  states  of  gates  m  and  n  relating  to  the 
membrane  potential  traces  shown  in  (A).  C:  High-gain  phase  space  of  spiking  and  non¬ 
spiking  regions  in  (B).  D:  The  initial  portion  in  (C)  is  shown  at  an  even  higher  gain  as  the  in 
m-n  phase  space  as  the  separatrix  between  trajectories  that  led  to  action  potentials  and  those 
that  do  not.  These  trajectories  (shown  by  the)  solid,  dotted,  and  small-dash  lines  are  these 
same  as  the  trajectories  labeled  a,b,c,  and  d  in  Fig.  10.  Integration  step  =  0.001  ms. 
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ABSTRACT.  We  have  examined  how  a  Hodgkin-Huxley  model  of  the  squid  axon 
membrane  responds  to  patterned  stimulation.  The  present  work  extends  the 
methods  reported  in  the  accompanying  paper  whereby  a  single,  low-level 
current  impulses  is  used  in  each  simulation  to  perturb  the  latency  of  action 
potentials  (spikes)  evoked  by  an  excitatory  postsynaptic  current  (EPSC).  A 
chaotic  logistic  function  controlled  the  presentation  time  (or  the  amplitude)  of  the 
impulse.  Over  a  number  of  simulations,  the  change  in  the  timing  of  the  impulse 
provided  temporally  structured  trains  of  stimuli  that  could  be  compared  to  the 
changes  in  the  latency  of  the  EPSC-evoked  spikes.  At  fixed  presentation  times, 
spike  latencies  were  found  to  be  a  linear  function  of  the  amplitude  of  the 
perturbing  impulse.  This  yielded  a  temporal  structure  in  the  sequence  of  spike 
latencies  that  was  similar  to  the  structure  in  the  sequence  of  different  impulse 
amplitudes.  Using  fixed  impulse  amplitudes  and  presenting  them  at  different 
times  produced  complex  sequences  of  spike  latencies.  The  output  pattern  was 
significantly  distorted  by  comparison  to  the  input  and  is  transformed  into  a 
higher  phase-space  dimension.  The  distortions  are  attributable  to  the  shape  of  the 
functions  relating  the  timing  of  the  perturbation  to  the  latency  of  the  action 
potential  which  we  examined  in  the  preceding  paper  with  respect  to  membrane 
impedance  under  perturbation.  Using  two  perturbing  impulses  in  each  sweep, 
one  having  a  fixed  timing  and  one  variable,  produced  amplitude  changes  in  the 
distortion  but  no  new  distortions.  Owing  to  the  sensitivity  of  the  membrane  to 
stimulus  patterns,  multiple,  temporally  varying  input  stimuli  would  produce 
complex  firing  patterns.  Together  the  findings  reported  here  and 
previously  provide  evidence  both  for  deterministic  and  nondeterministic  pattern 
detection  and  generation  in  single  neurons. 


1  Introduction 


A  large  body  of  research  has  indicated  that  structure  in  the  temporal  firing 
of  action  potentials  in  neurons  may  provide  a  basis  of  information  transfer 
between  neurons  rather  than,  or  in  addition  to,  other  response  characteristics, 
such  as  the  mean  firing  frequency,  e.g.,  (Adrian,  1928;  Diamond,  et  al.,  1988; 
Gerstein  and  Mandelbrot,  1964;  Glantz  and  Nudelman,  1976;  Glantz  and 
Nudelman,  1988;  Glantz,  et  al,  1984;  Griineis,  et  al.,  1989;  Nakahama,  et  al.,  1977; 
Perkel,  et  al.,  1964;  Rail,  1977;  Rose  and  Dunwiddie,  1986;  Segundo,  et  al.,  1963; 
Tsukada,  et  al.,  1994;  Yamamoto  and  Nakahama,  1983).  Along  this  line,  the  aim 
of  the  simulations  reported  in  the  preceding  paper  was  to  establish  a  foimdation 
defining  how  membrane  impedance,  acting  as  a  linear  filter,  might  provide  the 
mechanism  for  pattern  detection  and  generation  in  simple  neuron  membrane 
(Edstrom  and  Mpitsos,  1995).  We  examined  the  ability  of  a  simple  neuron  to 
generate  synaptically-evoked  action  potentials  under  conditions  when  the 
impedance  state  of  the  membrane  had  been  preconditioned  using  the  injection  of 
brief  low-level  current  impulses.  The  neuronal  membrane  characteristics  were 
defined  using  the  Hodgkin-Huxley  (1952)  parameters  for  sodium  and  potassium 
ion  channels.  An  excitatory  postsynaptic  current  (EPSC)  was  used  whose  effect 
was  simply  to  produce  smoothly  changing  unimodal  membrane  changes  having 
the  time  course  of  the  squid  giant  synapse.  The  temporal  history  of  impedance 
changes  that  occur  between  the  perturbing  impulse  and  the  EPSC-evoked  action 
potential  (spike)  had  complex  structure  that  resembled  i/ o  functions  obtained 
using  sub-  and  suprathreshold  white-noise  perturbation  methods  of  neurons  in  a 
number  of  animals  (Bryant  and  Segundo,  1976;  Guttman,  et  al.,  1974;  Sakuranaga 
and  Naka,  1983;  Sakuranaga  and  Naka,  1985a;  Sakuranaga  and  Naka,  1985b). 
The  findings  are  consistent  with  the  possibility,  also  proposed  in  animal  studies, 
that  membrane  may  a  act  as  a  linear  filter  for  pattern  detection  or  generation  e.g., 
(Bialek,  et  al.,  1993;  Bialek,  et  al.,  1991;  Enroth-Cugell,  et  al.,  1983;  Frishman,  et  al., 
1987;  Glantz,  1994;  Glantz,  et  al.,  1984;  Krause  and  Naka,  1980).  Our  findings  also 
suggested  that  the  same  i/o  functions  may  impose  a  form  of  degeneracy  in  the 
transfer  of  information  across  the  synapse. 

In  the  present  paper  we  extend  these  studies  to  show  that  the  postsynaptic 
neuron  responds  differentially  when  the  same  input  pattern  is  presented  at 
different  regions  of  the  temporal  span  of  the  i/ o  fimction.  However,  depending 
on  the  timing  of  the  input  stimuli,  the  output  spike  patterns  exhibit  distortions 


and  increases  in  complexity  by  comparison  the  same  measures  of  the  input 
pattern.  Such  differences  include  multiple  folding  regions  in  the  return  maps 
(phase-space  autocorrelations  of  one  spike  latency  against  the  preceding  one  in 
the  time  series)  that  require  increases  in  phase-space  embedding  to  resolve  some 
of  the  degeneracies.  The  aim  here  will  be  to  show  these  effects  graphically. 
Papers  that  follow  will  report  on  quantitative  measures  of  spike  trains  using 
information-theoretic  methods  and  symbolic  dynamics. 

2  Methods  and  Results 

Except  for  the  the  addition  of  a  chaotic  logistic  function  to  control  the 
presentation  of  the  perturbation  current  impulses,  methods  used  in  the  present 
series  of  experiments  were  the  same  as  those  reported  in  the  preceding  paper 
(Edstrom  and  Mpitsos,  1995).  A  two-cell  network  was  used  in  which  the 
presynaptic  neuron  activated  the  postsynaptic  by  means  of  an  EPSC.  A  single 
low-level  delta-function  current  impulses,  lasting  one  integration  step,  was 
presented  to  the  postsynaptic  cell  in  each  35  ms  simulation  sweep.  The  impulse 
was  presented  in  some  temporal  relationship  with  an  EPSC  whose  onset  was  at 
25  ms.  The  membrane  potential  was  reset  to  the  normal  resting  potential  after 
each  sweep.  The  impulses  were  well  below  the  firing  threshold  of  the  membrane, 
whereas  the  EPSC  was  suprathreshold  and  generated  a  single  action  potential.  In 
the  last  set  of  experiments  reported  here,  two  impulses  were  given,  one  acting  as 
a  preconditioning  stimulus  to  the  other.  In  both  sets  the  aim  is  to  examine  the 
effect  of  the  conditioning  impulses  on  the  latency  of  the  action  potential 
produced  by  the  EPSC. 

The  temporal  position  of  the  impulses  in  the  simulation  sweep  was 
controlled  either  by  sequences  of  computer  generated  random  numbers  (routine 
ranO  in  (Press,  et  al.,  1988,  pp.  207-208),  or  by  the  recursive  logistic  function  Xj+i  = 
R(l-Xi)Xi,  where  X  is  in  the  unit  interval.  For  R  =  3.7,  this  function  generates 
values  that  cover  the  unit  interval  densely  and  chaotically.  Both  sets  of  numbers 
were  scaled  so  as  to  cover  the  first  30  ms  of  the  simulation  sweeps,  or  some 
portion  of  them.  The  sum  of  many  such  simulation  sweeps  defines  input/output 
(i/o)  functions  that  relate  the  time  of  occurrence  of  the  input  perturbation 
impulse(s)  to  the  output  latency  generated  by  the  EPSC. 


2.1  The  perturbation  history  of  the  membrane  imposes  structure  on  randomly  spaced 
stimuli 

An  example  of  an  i/o  function  is  shown  in  Fig.  lA.  The  time-axis  of  the 
simulation  sweeps  rises  vertically,  and  the  associated  latencies  produced  by  the 
EPSC  are  on  the  horizontal  axis.  One  perturbation  current  impulse  ( 5  x  10*10 
Amps)  was  randomly  timed  to  occur  within  the  7-30  ms  portion  of  each 
simulation  sweeps.  Earlier  presentation  times  were  not  used  because  at  the 
normal  resting  potential,  small  perturbation  currents  produce  little  effect  on  the 
spike  latencies.  A  total  of  100,000  sweeps  were  rtm  in  which  the  random  signals 
covered  the  7-30  ms  interval  homgeneously  (Fig.  IB).  This  generated  100,000 
action-potential  latencies  having  a  non-homogeneous  frequency  distribution  (Fig. 
1C),  showing  that  the  membrane  imposes  structure  on  the  input  data  stream  that 
it  receives. 

2.1  The  perturbation  history  of  the  membrane  distorts  and  adds  complexity  to  the 
temporal  structure  of  the  signals  that  it  receives 

The  chaotic  logistic  shiftes  the  timing  of  the  perturbing  impulse  back  and 
forth  with  respect  to  the  occurrence  of  the  EPSC.  Being  deterministic,  the  logistic 
provides  a  well-defined  pattern  of  shifts  that  can  be  compared  with  the  pattern 
of  latencies  in  the  output  spike  train.  Figure  2A  shows  a  sample  time  series. 
Figure  2B  shows  the  structure  of  the  time  series  as  the  return  map,  where  a  given 
interval  (Xi+i)  is  mapped  against  the  interval  preceding  it  (Xi).  For  reference,  the 
first  four  points  are  mapped  in  the  return  map  with  the  same  symbols  as  the  first 
four  points  in  the  time  series. 


FIGURE  2  HERE 

In  some  experiments,  we  held  the  presentation  time  constant  and  used  the 
logistic  to  control  the  amplitude  of  the  impulses  using  Lj  =  fX/  -  0.5)  x  10  Amp. 
In  all  cases  the  impulse  width  was  the  same  as  the  integration  time  step  (usually 
0.010  msec).  In  other  experiments  we  held  the  impulse  amplitude  constant  and 
varied  the  presentation  times  T  using  Tj  =  C  +  (Xi  -  0.5)D,  where  Xf  is  the  ouput 
of  the  logistic  equation.  The  constant  D  controls  the  range  of  the  logistic  values, 
and  C  slides  the  critical  point  of  this  range  along  the  time  axis.  The  value  of  the 


critical  point  in  the  logistic  itself  is  0.5,  and  is  the  center  of  folding  in  the  return 
map  by  which  the  points  mix  with  respect  to  one  another,  as  shown  in  Fig.  2B 
(and  see  (Devaney,  1986;  May,  1976)).  The  i/o  function  representing  the  complete 
set  of  latencies  obtained  in  the  experiment  is  shown  by  the  curve  in  Fig.  3C. 

Using  the  45°  line  in  Fig.  3B,  it  can  be  seen  how  the  time  series  of  current 
impulses  (Fig.  3A)  are  mapped  back  and  forth  onto  the  temporal  position  of 
successive  simulation  sweeps  (Fig.  3C).  The  symbols  in  Fig.  3  A-D  identify  each 
impulse  with  the  corresponding  latency  in  the  output  time  series  (Fig.  3D). 

FIGURE  3  HERE 

When  holding  the  presentation  times  constant,  the  changes  in  the  latency 
are  nearly  a  constant  function  of  amplitude  (Fig.  4B),  and  the  return  maps  of  the 
latencies,  constructed  by  plotting  the  latency  obtained  in  one  simulation  sweep 
against  the  latency  in  the  preceding  sweep,  are  quite  similar  to  the  return  map  of 
the  logistic-generated  amplitudes  (compare  Fig.  4 A  with  Fig.  2B).  Minor 
stretching  and  compression  may  occur  in  the  latency  return  maps  at  other 
constant  presentation  times  along  the  isopleths,  but  overall  their  shape  is  similar 
to  the  shape  shown  in  Fig.  2B  for  the  input  logistic  function. 

Holding  the  amplitude  constant  and  presenting  the  impulses  at  different 
times  produces  complex  changes  in  the  spike  latencies.  The  results  of  two 
experiments  are  in  Fig.  4D,  where  the  two  sets  of  impulses  range  over  different 
times  and  have  their  critical  points  shifed  with  respect  to  one  another  in  the 
simulation  sweep.  The  latencies  arising  from  impulses  in  one  experiment  are 
shown  by  open  circles,  and  those  from  the  other  are  shown  by  small  dots  that 
coallesce  into  a  line  at  this  resolution.  The  utility  of  using  the  logistic  to  control 
the  impulse-presentation  times  becomes  evident  when  plotting  the  latency 
changes  as  return  maps  in  Fig.  4C.  Both  return  maps  have  two  folding  regions, 
wheres  the  input  logistic  has  only  one.  The  effect  of  changing  the  position  and 
range  of  the  impulses  is  shown  more  fully  in  Fig.  5.  The  thin,  unbroken  line  in 
Fig.  5A  is  a  fuller  representation  of  the  standard  i/o  function  obtained  from 
membrane  at  resting  potential  using  lO'iO  Amp  impulses  shown  in  Fig.  4D.  The 
presentation  times  of  seven  different  sets  of  input  stimulus  trains  are  shown  by 
the  filled  circles;  they  are  shifted  vertically  with  respect  to  one  another  in  order  to 
illustrate  their  positions.  Figure  5B  shows  the  return  maps  of  the  spike  latencies; 
each  map  is  identified  numerically  by  the  range  of  the  input  stimulus  times  that 


generated  it.  Like  the  findings  in  Fig.  4C,  the  return  maps  are  significantly 
distorted  variously  by  comparison  to  the  input  stimulus  pattern  often  resulting  in 
more  than  one  folding  region  and  even  in  ones  having  crossed  trajectories  (Fig. 

5B  "14-19").  Crossed  trajectories  occur  whenever  the  stimulus  pattern  sufficiently 
spans  a  humped  (vertically  convex  or  concave)  region  of  the  i/ o  functions.  Four 
of  these  regions  are  shown  in  Fig.  6A,  two  each  for  depolarizing  and 
hyperpolarizing  impulses.  The  corresponding  return  maps  are  in  Fig.  6B. 
Together  the  findings  in  Figs.  4(C  and  D),  5B,  and  6  indicate  that  the 
conditioning  history,  introduced  by  a  single  low-level  current  impulse, 
significantly  distorts  or  increases  the  complexity  of  the  spike-latency  return  maps 
obtained  in  response  to  a  test  EPSC. 

In  biological  systems,  neurons  are  bombarded  by  many  synaptic  events. 
These  conditions  are  not  controllable  in  biological  systems  to  obtain  the  kind  of 
perturbation  studies  we  have  attempted  here,  though  they  may  be  approximated 
in  simulations  (Mpitsos  and  Edstrom,  1995).  In  the  experiments  presented  up  to 
now  in  the  present  and  in  the  preceding  paper,  we  have  used  only  one 
conditioning  impulse  and  one  test  EPSC  in  each  simulation  sweep.  Between 
sweeps  the  membrane  was  returned  to  the  normal,  unperturbed  resting 
potential.  A  step  toward  creating  controlled,  multiple  stimulation  conditions  may 
be  made  by  presenting  two  conditioning  pulses  in  conjunction  with  the  EPSC. 
The  general  design  of  these  experiments  is  as  follows:  A  first  conditioning 
impulse  was  presented  at  some  fixed  position  in  each  simulation  sweep.  This 
impulse  preconditioned  the  state  of  the  membrane  to  the  arrival  of  a  second 
conditioning  impulse.  The  timing  of  the  second  impulse  was  controlled  by  the 
chaotic  logistic;  as  before,  the  use  of  the  logistic  provides  a  structured  train  of 
stimuli  from  one  sweep  to  the  next  that  can  be  used  to  assess  the  effect  of  the 
stimulus  environment  on  the  output  of  the  postsynaptic  neuron. 

Figure  7A  provides  an  example  of  one  such  experiment.  The  fixed 
impulse,  lasting  one  integration  step,  was  given  at  19  ms  in  the  simulation  sweep. 
In  one  experiment  the  amplitude  of  this  impulse  was  lO'l®  Amps  (open  circle);  in 
a  second  experiment  its  amplitude  was  -10*^0  Amps  (filled  circles).  The  dashed 
lines  running  through  these  points  indicate  the  latency  of  the  action  potential 
obtained  when  only  this  first  impulse  was  paired  with  the  EPSC.  The  long- 
dashed  line  lying  between  these  indicates  the  latency  obtained  when  presenting 
only  the  EPSC.  In  order  to  provide  a  reference  for  the  results  obtained  using  the 
second  im  pulse  that  was  controlled  by  the  logistic  function.  Fig.  7A  contains  i/o 


functions  (solid-line  curves)  in  which  the  position  of  the  second  impulse  is 
moved  successively  along  the  time  axis  in  each  sweep  from  left  to  right,  as  was 
done  to  generate  Fig.  10  of  the  preceding  paper.  There  are  two  sets  of  solid-line 
curves  for  each  experiment,  one  in  which  the  amplitude  of  the  second  impulse 
was  4  X  lO'lO  Amps  and  one  in  which  it  -4  x  lO-^O  Amps;  in  both  cases  each  of 
these  movable  impulses  was  paired  with  one  of  the  fixed  Amp  impulses 
and  the  EPSC.  The  effect  of  the  fixed-position  impulse  is  to  shift  the  two  sets  of 
curves  vertically,  one  set  above  the  control  position  shown  by  long-dashed  line, 
and  one  set  below;  i.e.,  the  fixed-position  impulse  acts  as  bias  on  the  membrane 
potential,  or  equivalently,  on  the  state  of  the  gate  parameters.  A  third  set  of 
experiment  is  superimposed  on  these  results  by  the  thickened  region  of  the  i/  o 
curves  obtained  using  the  4  xlO'lO  Amp  impulses.  In  this  set  of  experiments,  the 
fixed  impulses  were  used  in  conjunction  with  the  EPSC,  but  the  timing  of  the  4  x 
lO'lO  Amp  impulses  was  controlled  by  the  logistic  over  the  simulation  times  that 
are  indicated  by  the  thickened  curves. 

The  return  maps  of  the  latencies  delimited  by  the  thickened  curves  in  Fig. 
7A  are  plotted  in  Fig.  7B,  showing  that  the  depolarizing  preconditioning  states 
produced  by  the  fixed  impulse  expand  the  maps  whereas  hyperpolarizing  shrink 
them.  Thus,  adding  the  fixed  impulse  produced  no  new  changes  in  the  maps 
except  for  amplification,  and,  since  the  fixed  impulses  shifted  the  i/ o  fimctions 
vertically,  the  two  return  maps  were  shifted  to  different  regions  of  phase  space. 
The  changes  in  amplification,  however,  would  further  complicate  the  return 
maps  if  both  impulses  had  variable  timing. 

3  Discussion 


As  suggested  by  a  number  of  studies  in  biological  systems,  noted  above 
and  reviewed  in  the  accompanying  paper,  the  findings  reported  here  illustrate 
that  the  membrane  acts  as  an  active  linear  filter,  responding  differentially  to  the 
same  stimulus  pattern  presented  at  different  times,  and  adding  structure  to 
randomly  spaced  input  signals.  The  output  of  the  cell  represents  a  distortion  of 
the  input,  producing  several  folding  regions  in  return  maps  and  changing  the 
shape  and  amplitude  of  the  input  pattern. 

Overall  the  output  represents  an  increase  in  complexity  with  respect  to  the 
input  pattern,  since  a  single-folded  return  map  of  the  chaotic  logistic  that  is 
characterizable  as  by  a  second  order  polynomial,  is  converted  to  a  doubly-folded 


map  requiring  a  higher  order  polynomial  to  define  it.  The  input  stimulus  is  a  1- 
dimensional  process,  and  the  phase-space  dynamics  of  the  input  pattern  can  be 
characterized  accurately  by  simple  2-dimensional  return  maps  (Fig.  2B)  that 
represent  autocorrelations  between  adjacent  input  values.  The  inter-spike 
intervals  are  also  a  1-dimensional  process,  but  under  some  impulse  presentation 
timing,  the  membrane  forces  the  output  dynamics  into  higher  phase-space 
dimensions  than  the  input  since  autocorrelations  between  several  intervals  in 
succession  are  required  to  resolve  the  dynamics.  These  instances  occur  when  the 
return  maps  have  crossed  ends  (Fig.  5B  ("14-19"),  Fig.  6B,  and  Fig.  7B).  Owing  to 
the  Poincare-Bendixson  theorem  (Hirsch  and  Smale,  1974)  and  the  Jordon  curve 
theorem,  phase-space  trajectories  can  not  cross  in  deterministic  systems. 
Although  it  is  not  shown  here,  casting  the  return  maps  in  3-dimensions  (Xn, 

Xn+i,  Xn+3)  and  rotating  the  image,  the  trajectories  do  not  cross,  showing  that  a 
changing  the  reference  from  a  2-dimensional  return  map  to  a  3-dimensional  one 
resolves  at  least  this  degeneracy  in  the  input/output  transformation,  though 
higher  dimensional  embedding  may  be  necessary  to  characterize  the  dynamics 
accurately. 

Two  important  and  possibly  interrelated  phenomena  have  been  identified 
in  our  studies  that  may  be  useful  in  extending  the  imderstanding  of  how 
individual  neurons  may  become  involved  in  the  integrative  capabilities  of 
networks.  Both  emerge  from  the  input/output  functions  relating  the  linear 
perturbation  history  of  the  membrane  impedance.  One,  reported  in  the 
accompanying  paper,  is  the  apparent  degeneracy  of  how  information  in  the  input 
data  stream  may  relate  or  be  decipherable  in  the  output  data  stream.  The 
second,  reported  here,  shows  that  that  there  is  also  a  deterministic  component, 
though  there  is  distortion  and  increase  in  complexity  of  the  output  firing  pattern 
by  comparison  to  the  input  signal. 

These  findings  speak  to  both  sides  of  a  controversy  that  has  had  a  long 
existence,  e.g.,  (Adey,  1972;  Adrian,  1946;  John,  1972).  To  use  Adey’s  words,  are 
neurons  inefficient,  "noisy  processor"  or  do  they  process  information  efficiently 
and  deterministically?  Recent  work  on  hippocampal  slices  has  obtained  evidence 
for  determinism  (Schiff,  1994),  but  also  for  a  substantial,  perhaps  in  most  cases,  of 
nondeterministic  activity  (Schiff,  et  al.,  1994).  The  problem  has  been  one  not  only 
of  neuronal  function  in  mammals,  but  also  one  of  "simple"  function  in  lower 
animals  (Mpitsos  and  Cohan,  1986;  Mpitsos,  et  al.,  1988b;  Wu,  et  al.,  1994).  In  our 
own  biological  work  (Mpitsos,  1989;  Mpitsos,  et  al.,  1988a;  Mpitsos,  et  al.,  1988b), 
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the  search  for  low-dimensional  deterministic  activity  has  not  been  encouraging, 
and  has  prompted  the  present  simulations  to  gain  some  understanding  of  where 
the  apparent  nondeterministic  activity  arises.  In  a  commentary  on  a  deterministic 
model  proposed  by  Shalden  and  Newsome  (1994),  Softky  (1995)  has  discussed 
that  their  model  of  ’balanced  inhibition',  used  to  account  for  variations  in  cortical 
neuron  firing,  is  based  on  the  assumption  that  the  cell  is  insensitive  to  the 
temporal  structure  of  the  input  signal.  Our  findings  indicate  that  any  neuron 
having  complex,  humped  or  multi-humped  i/o  functions  will  be  highly  sensitive 
to  the  temporal  structure  of  its  input  signal.  Although  the  process  is 
deterministic,  it  is  also  highly  subject  to  degeneracies  that  can  lead  to  variations. 
Our  subsequent  papers  will  examine  how  the  humped  i/o  functions  lead  both  to 
stable  and  unstable  firing  when  the  neuron  is  placed  in  a  network,  and  will 
examine  the  information  transfer  quantitatively. 
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FIGURE  LEGENDS 


FIGURE  1:  Recent  history  of  impedance  changes  that  occur  under  random 
perturbation  generates  nonrandom  firing  of  EPSC-evoked  action  potentials.  A: 
Relation  of  impulse  presentation  time  to  action  potential  latency  (horizontal  axis). 
B:  Density  of  randomly  generated  impulse  presentation  times;  vertical  axis  is  the 
same  as  in  (A).  A  single  randomly  timed  current  impulse  (4  x  10*10  Amps,  lasting 
one  .010  ms  integration  step)  was  presented  in  the  7-30  ms  interval  of  each  35  ms 
simulation  sweep.  A  suprathreshold  EPSC  was  initiated  25  ms  into  the 
simulation.  C:  Histogram  of  the  EPSC-evoked  action  potentials  the  resulted  from 
100,000  simulation  sweeps. 

FIGURE  2;  The  chaotic  logistic,  f(x)  =  3.7(l-x)x  was  used  to  generate  deterministic 
sequences  of  impulse  presentation  times.  A:  Time  series.  The  values  were  scaled 
(see  text)  to  control  the  impulse  presentation  times  within  each  simulation 
sweep.  B:  Return  map  showing  the  deterministic  structure  of  the  logistic.  One 
value  (Xi+i)  of  the  time  series  is  plotted  against  the  preceding  value  (Xi).  For 
reference,  the  symbols  are  the  same  in  (A  and  B)  for  the  first  four  points.  The  1:1 
line  shows  how  the  points  fold  over  the  critical  point  (X  =  .5). 

FIGURE  3:  Method  for  presenting  the  logistically-driven  constant  current  pulses 
to  the  postsynaptic  neuron  variably  in  time.  A:  Logistic-driven  current  pulses.  B: 
the  1:1  line  is  used  to  match  time  axes  so  that  the  current  pulses  can  be 
superimposed  on  the  time  axis  of  the  isopleth  (C)  to  show  how  they  bounce  back 
and  forth  in  time.  D:  Symbols  show  the  observed  EPSC-evoked  action  potential 
latencies  arising  from  each  pairing  of  the  impulse  with  the  EPSC. 


FIGURE  4:  The  effect  of  the  amplitude  (A  and  B)  and  timing  (C  and  D)  of 
perturbation  pulses  on  EPSC-evoked  action-  potential  latencies.  A  and  C.  return 
maps  of  the  latencies.  B  and  D:  latencies  as  a  function  of  current-pulse  amplitude 
and  time  of  occurrence,  respectively.  For  each  presentation  of  a  0.010  msec 
perturbation  pulse  and  an  EPSC,  a  single  action  potential  was  obtained  whose 
latency  was  measured  as  the  time  from  the  onset  of  the  sunulation.  Both 
amplitude  and  time  were  modulated  by  the  3.7  logistic.  When  changing  the 
presentation  time,  the  amplitude  was  constant  at  2  x  10  1®  A.  The  presentation 
times  for  two  experiments  are  shown  min  C  and  D:  The  solid  line  is  for 


presentation  times  shifted  earlier  and  covered  a  smaller  range  than  the  times 
shown  for  the  experiment  shown  by  the  open  circles.  The  EPSC  onset  was  at  25 
msec 

FIGURE  5:  The  temporal  position  along  the  latency  isopleths  represent  different 
distortions  of  the  structure  of  the  input  fimction  generating  the  perturbation 
pulses.  A:  The  thin  line  shows  the  action  potential  latencies  arising  from  a 
composite  of  logistic-activated  sequence  of  current  pulses  having  constant 
amplitude  (9  x  lO'lO  Amp;  .010  ms).  Seven  of  these  sequences  are  shown  by  the 
dotted  curves  which  have  been  shifted  vertically  to  identify  them.  The  irregular 
spacing  between  the  dots  in  each  line  arises  from  the  logistic  scaling  function  T 
described  in  the  text.  B:  Return  maps  of  the  intervals  in  sb<  of  the  sequences  in 
(A).  The  numbers  identify  the  approximate  range  of  times  spanned  by  each 
sequence  along  the  isopleth.  Note  that  each  map  is  a  different  distortion  of  the 
input  function  return  map  shown  in  Fig.  8,  and,  in  particular,  that  the  two  legs  of 
the  map  for  the  presentation  of  the  input  function  at  the  14-19  msec  span  on  the 
isopleth  cross  over  one  another. 

FIGURE  6:  Trains  of  stimuli  falling  over  four  humped  regions  (normal  and 
inverted)  of  the  isopleths  (A)  produce  different  looped  latency  return  maps  (B). 
The  two  smaller  humps  occur  before  the  onset  of  the  EPSC.  The  second  two 
partially  overlap  the  EPSC,  showing  that  the  EPSC  amplifies  the  latency  changes 
by  comparison  to  the  latencies  produced  by  stimuli  falling  over  the  first  set  of 
humps,  but  does  not  change  the  overall  structure  of  the  maps. 

FIGURE  7:  Interaction  between  multiple  perturbation  impulses  amplifies 
changes  in  EPSC-evoked  latencies,  but  does  not  change  overall  output  dynamics. 
A:  To  show  the  effect,  three  experiments  were  conducted  involving  two 
conditioning  impulses  and  a  suprathreshold  EPSC).  In  two  experiments  (two 
sets  of  thin-line  curves),  a  fixed  conditioning  pulse  was  presented  at  19  ms  in  the 
simulation  sweep  (filled  circle  =  lO'l®  Amps;  open  circle  =  -10"^®  Amps).  The 
timing  of  the  second  impulse  (4  xlO'^O  or  4  x-lO'^O^Ps)  was  moved 
progressively  from  left  to  right  in  each  simulation  sweep,  and  enough  sweeps 
were  conducted  to  cover  the  full  sweep.  To  show  the  shapes  of  the  i/ o  curves, 
only  the  15-30  msec  range  of  times  is  shown.  The  third  experiment  paired  the 
fixed-position  impulses  with  the  movable  impulse  whose  timing  was  controlled 


by  the  logistic  to  occur  over  the  range  shown  by  the  thickened  line  segments.  B: 
Return  maps  of  the  latencies  shown  by  the  two  thickened  lines.  Not  that  the  two 
maps  are  simple  linear  amplifications  of  one  another.  Middle,  long-dashed  line, 
latency  obtained  with  only  the  EPSC.  Top  dashed  line:  latency  obtained  by 
pairing  the  10'^®  Amp  impulse  with  the  EPSC.  Bottom  dashed  line:  latency 
obtained  by  pairing  the  -IQ-IO  Amp  impulse  with  the  EPSC. 
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